Duration perception in crossmodally-defined intervals

How humans perform duration judgments with multisensory stimuli is an ongoing debate. Here, we investigated how sub-second duration judgments are achieved by asking participants to compare the duration of a continuous sound to the duration of an empty interval in which onset and offset were marked by signals of different modalities using all combinations of visual, auditory and tactile stimuli. The pattern of perceived durations across five stimulus durations (ranging from 100 ms to 900 ms) follows the Vierordt Law. Furthermore, intervals with a sound as onset (audio-visual, audio-tactile) are perceived longer than intervals with a sound as offset. No modality ordering effect is found for visualtactile intervals. To infer whether a single modality-independent or multiple modality-dependent time-keeping mechanisms exist we tested whether perceived duration follows a summative or a multiplicative distortion pattern by fitting a model to all modality combinations and durations. The results confirm that perceived duration depends on sensory latency (summative distortion). Instead, we did not find evidence for multiplicative distortions. The results of the model and the behavioural data support the concept of a single time-keeping mechanism that allows for judgments of durations marked by multisensory stimuli.     

Auditory and visual differences in time perception

The present experiment assessed intersensory differences in temporal judgments, that is, auditory stimuli are perceived as longer than physically equivalent visual stimuli. The results confirmed the intersensory difference. Auditorially defined intervals were experienced as longer than visually defined intervals. Auditory boundaries were perceived as longer than visual ones. An interaction of boundary modality and interval modality was obtained which suggested that auditorially defined intervals provided more temporal information about events occurring in close temporal proximity than visually defined intervals. It was hypothesized that cognitive factors, specifically stimulus complexity, would affect the auditory and visual systems differentially. This hypothesis was not substantiated, although highly complex stimuli were experienced as longer than those of low complexity.     

Auditory/visual duration bisection in patients with left or right medial-temporal lobe resection

Patients with unilateral (left or right) medial temporal lobe lesions and normal control (NC) volunteers participated in two experiments, both using a duration bisection procedure. Experiment 1 assessed discrimination of auditory and visual signal durations ranging from 2 to 8 s, in the same test session. Patients and NC participants judged auditory signals as longer than equivalent duration visual signals. The difference between auditory and visual time discrimination was equivalent for the three groups, suggesting that a unilateral temporal lobe resection does not modulate the modality effect. To document interval-timing abilities after temporal lobe resection for different duration ranges, Experiment 2 investigated the discrimination of brief, 50-200 ms, auditory durations in the same patients. Overall, patients with right temporal lobe resection were found to have more variable duration judgments across both signal modality and duration range. These findings suggest the involvement of the right temporal lobe at the level of the decision process in temporal discriminations.     

On the interplay of visuospatial and audiotemporal dominance: Evidence from a multimodal kappa effect

When participants judge multimodal audiovisual stimuli, the auditory information strongly dominates temporal judgments, whereas the visual information dominates spatial judgments. However, temporal judgments are not independent of spatial features. For example, in the kappa effect, the time interval between two marker stimuli appears longer when they originate from spatially distant sources rather than from the same source. We investigated the kappa effect for auditory markers presented with accompanying irrelevant visual stimuli. The spatial sources of the markers were varied such that they were either congruent or incongruent across modalities. In two experiments, we demonstrated that the spatial layout of the visual stimuli affected perceived auditory interval duration. This effect occurred although the visual stimuli were designated to be task-irrelevant for the duration reproduction task in Experiment 1, and even when the visual stimuli did not contain sufficient temporal information to perform a two-interval comparison task in Experiment 2. We conclude that the visual and auditory marker stimuli were integrated into a combined multisensory percept containing temporal as well as task-irrelevant spatial aspects of the stimulation. Through this multisensory integration process, visuospatial information affected even temporal judgments, which are typically dominated by the auditory modality.     

Does time-shrinking take place in visual temporal patterns?

The duration of a short empty time interval (typically shorter than 300 ms) is often underestimated when it is immediately preceded by a shorter time interval. This illusory underestimation--time-shrinking--had been studied only with auditory temporal patterns. In the present study, we examined whether similar underestimation would take place with visual temporal patterns. It turned out that underestimation of the same kind takes place also in the visual modality. However, a considerable difference between the auditory and the visual modalities appeared. In the auditory modality, it had been shown that the amount of underestimation decreased for preceding time intervals longer than 200 ms. In the present study, the underestimation increased when the preceding time interval varied from 160 to 400 ms. Furthermore, the differences between the two neighbouring intervals which could cause this underestimation had always been in a fixed range in the auditory modality. In the visual modality, the range was broader when the intervals were longer. These results were interpreted in terms of an assimilation process in light of the processing-time hypothesis proposed by Nakajima (1987 Perception 16 485-520) in order to explain an aspect of empty-duration perception.     

Crossmodal duration perception involves perceptual grouping, temporal ventriloquism, and variable internal clock rates

Here, we investigate how audiovisual context affects perceived event duration with experiments in which observers reported which of two stimuli they perceived as longer. Target events were visual and/or auditory and could be accompanied by nontargets in the other modality. Our results demonstrate that the temporal information conveyed by irrelevant sounds is automatically used when the brain estimates visual durations but that irrelevant visual information does not affect perceived auditory duration (Experiment 1). We further show that auditory influences on subjective visual durations occur only when the temporal characteristics of the stimuli promote perceptual grouping (Experiments 1 and 2). Placed in the context of scalar expectancy theory of time perception, our third and fourth experiments have the implication that audiovisual context can lead both to changes in the rate of an internal clock and to temporal ventriloquism-like effects on perceived on- and offsets. Finally, intramodal grouping of auditory stimuli diminished any crossmodal effects, suggesting a strong preference for intramodal over crossmodal perceptual grouping (Experiment 5).     

Markers’ influence on the duration discrimination of intermodal intervals

The effects of sensory signal characteristics on the duration discrimination of intermodal intervals was investigated in three experiments. Temporal intervals were marked by either the successive presentation of a visual then auditory signal (VA), or by the successive presentation of an auditory then visual signal (AV). The results indicated that (1) VA intervals are generally easier to discriminate than are AV intervals, but this effect depends on the range of duration studied; (2) AV intervals are perceived as longer than VA intervals for durations ranging from 250 to 750 msec; (3) the intensity of the visual markers for both AV and VA intervals does not affect the discrimination; and (4) the perceived duration of an intermodal interval is influenced by the length of the first and second markers. The results are mainly interpreted in terms of (1) a sensory trace left by visual and auditory signals and (2) the detection of these signals.     

Accuracy of temporal coding: Auditory-visual comparisons

Three experiments were designed to decide whether temporal information is coded more accurately for intervals defined by auditory events or for those defined by visual events. In the first experiment, the irregular-list technique was used, in which a short list of items was presented, the items all separated by different interstimulus intervals. Following presentation, the subject was given three items from the list, in their correct serial order, and was asked to judge the relative interstimulus intervals. Performance was indistinguishable whether the items were presented auditorily or visually. In the second experiment, two unfilled intervals were defined by three nonverbal signals in either the auditory or the visual modality. After delays of 0, 9, or 18 sec (the latter two filled with distractor activity), the subjects were directed to make a verbal estimate of the length of one of the two intervals, which ranged from 1 to 4 sec and from 10 to 13 sec. Again, performance was not dependent on the modality of the time markers. The results of Experiment 3, which was procedurally similar to Experiment 2 but with filled rather than empty intervals, showed significant modality differences in one measure only. Within the range of intervals employed in the present study, our results provide, at best, only modest support for theories that predict more accurate temporal coding in memory for auditory, rather than visual, stimulus presentation.     

The structure of sensory events and the accuracy of time judgments

We investigated how does the structure of empty time intervals influence temporal processing. In experiment 1, the intervals to be discriminated were the silent durations marked by two sensory signals, both lasting 10 or 500 ms; these signals were two identical flashes (intramodal: VV), or one visual flash (V) followed by an auditory tone (A) (intermodal: VA). For the range of duration under investigation (standards = 0.2, 0.6, 1, or 1.4 s), the results indicated that both the marker length and sensory mode influenced discrimination, but no interaction between these variables or between one of these variables and standard duration was significant. In experiment 2, we compared, for each of four marker-type conditions (VV, AA, VA, AV; and standard = 1 s), intervals marked by two 10 ms signals with intervals marked by unequal signal length (markers 1 and 2 lasting 10 and 500 ms, or 500 and 10 ms). As in experiment 1, the results revealed significant marker-mode and marker-length effects, but no significant interaction between these variables. Experiment 3 showed that, for the same conditions as in experiment 2, perceived duration is not influenced by marker length and that the variability of interval reproductions does not depend on the perceived duration of intervals. The results are discussed in the light of a single-clock hypothesis: marker-length and marker-mode effects are presented as being non-temporal sources of variability associated mainly with sensory and memory processes.     

Modality-specific preparatory influences on the flexibility of cognitive control in task switching

In the current study, we addressed modality-specificity of the flexibility of cognitive control. We compared performance in single-task and mixed-tasks blocks between blocked auditory and visual stimuli assessing alternation costs (single vs. mixed). Mixed blocks comprised task switches only. The tasks consisted of numerical parity, magnitude, and distance judgments about numbers between one and nine without five. A cue indicated the relevant task. The cue–stimulus interval was varied (short vs. long interval) to examine preparation effects. The results indicated higher response times (RTs) and error rates (ERs) in mixed- vs. single-tasks blocks. The alternation costs in ERs were larger for auditory compared to visual stimulus presentation. Moreover, the reduction of RT alternation costs based on increased preparation time was more pronounced for the auditory modality compared to the visual modality. These results suggest a modality-specific influence on processes involved in maintaining and updating task sets in working memory.     

Perceptual learning in auditory temporal discrimination: No evidence for a cross-modal transfer to the visual modality

Perceptual learning was used to study potential transfer effects in a duration discrimination task. Subjects were trained to discriminate between two empty temporal intervals marked with auditory beeps, using a twoalternative forced choice paradigm. The major goal was to examine whether perceptual learning would generalize to empty intervals that have the same duration but are marked by visual flashes. The experiment also included longer intervals marked with auditory beeps and filled auditory intervals of the same duration as the trained interval, in order to examine whether perceptual learning would generalize to these conditions within the same sensory modality. In contrast to previous findings showing a transfer from the haptic to the auditory modality, the present results do not indicate a transfer from the auditory to the visual modality; but they do show transfers within the auditory modality.     

Audition dominates vision in duration perception irrespective of salience,attention, and temporal discriminability

Whereas the visual modality tends to dominate over the auditory modality in bimodal spatial perception, the auditory modality tends to dominate over the visual modality in bimodal temporal perception. Recent results suggest that the visual modality dominates bimodal spatial perception because spatial discriminability is typically greater for the visual than for the auditory modality; accordingly, visual dominance is eliminated or reversed when visual-spatial discriminability is reduced by degrading visual stimuli to be equivalent or inferior to auditory spatial discriminability. Thus, for spatial perception, the modality that provides greater discriminability dominates. Here, we ask whether auditory dominance in duration perception is similarly explained by factors that influence the relative quality of auditory and visual signals. In contrast to the spatial results, the auditory modality dominated over the visual modality in bimodal duration perception even when the auditory signal was clearly weaker, when the auditory signal was ignored (i.e., the visual signal was selectively attended), and when the temporal discriminability was equivalent for the auditory and visual signals. Thus, unlike spatial perception, where the modality carrying more discriminable signals dominates, duration perception seems to be mandatorily linked to auditory processing under most circumstances.     

Differential effects of auditory and visual signals on clock speed and temporal memory

The effects of signal modality on duration classification in college students were studied with the duration bisection task. When auditory and visual signals were presented in the same test session and shared common anchor durations, visual signals were classified as shorter than equivalent duration auditory signals. This occurred when auditory and visual signals were presented sequentially in the same test session and when presented simultaneously but asynchronously. Presentation of a single modality signal within a test session, or both modalities but with different anchor durations did not result in classification differences. The authors posit a model in which auditory and visual signals drive an internal clock at different rates. The clock rate difference is due to an attentional effect on the mode switch and is revealed only when the memories for the short and long anchor durations consist of a mix of contributions from accumulations generated by both the fast auditory and slower visual clock rates. When this occurs auditory signals seem longer than visual signals relative to the composite memory representation.     

Time perception,attention, and memory: A selective review

This article provides a selective review of time perception research, mainly focusing on the authors' research. Aspects of psychological time include simultaneity, successiveness, temporal order, and duration judgments. In contrast to findings at interstimulus intervals or durations less than 3.0–5.0 s, there is little evidence for an “across-senses” effect of perceptual modality (visual vs. auditory) at longer intervals or durations. In addition, the flow of time (events) is a pervasive perceptual illusion, and we review evidence on that. Some temporal information is encoded All rights reserved. relatively automatically into memory: People can judge time-related attributes such as recency, frequency, temporal order, and duration of events. Duration judgments in prospective and retrospective paradigms reveal differences between them, as well as variables that moderate the processes involved. An attentional-gate model is needed to account for prospective judgments, and a contextual-change model is needed to account for retrospective judgments.     

自我损耗的视听通道效应

采用双任务实验范式,通过两个实验探讨了自我损耗的通道效应。实验1考察了完成视听工作记忆任务对执行手柄任务（非视听任务）的影响,结果发现,完成视觉损耗任务导致手柄任务的成绩显著地变差,完成听觉损耗任务、视听非损耗任务均对手柄任务的成绩没有影响。实验2考察了完成视听工作记忆任务对执行Stroop任务（视觉任务）的影响,结果发现,完成视觉损耗任务导致Stroop任务的成绩显著地变差,完成听觉损耗任务、视听非损耗任务对Stroop任务的成绩没有影响。研究结果表明同一自我控制任务通过视听通道完成对随后另一自我控制任务的影响不同,证实了自我损耗的视听通道效应。     

Auditory and visual differences in time perception? An investigation from a developmental perspective with neuropsychological tests

Adults and children (5- and 8-year-olds) performed a temporal bisection task with either auditory or visual signals and either a short (0.5-1.0s) or long (4.0-8.0s) duration range. Their working memory and attentional capacities were assessed by a series of neuropsychological tests administered in both the auditory and visual modalities. Results showed an age-related improvement in the ability to discriminate time regardless of the sensory modality and duration. However, this improvement was seen to occur more quickly for auditory signals than for visual signals and for short durations rather than for long durations. The younger children exhibited the poorest ability to discriminate time for long durations presented in the visual modality. Statistical analyses of the neuropsychological scores revealed that an increase in working memory and attentional capacities in the visuospatial modality was the best predictor of age-related changes in temporal bisection performance for both visual and auditory stimuli. In addition, the poorer time sensitivity for visual stimuli than for auditory stimuli, especially in the younger children, was explained by the fact that the temporal processing of visual stimuli requires more executive attention than that of auditory stimuli.     

Equivalent inter- and intramodality long-term priming: Evidence for a common lexicon for words seen and words heard

Weaker inter- than intramodality long-term priming of words has promoted two hypotheses: (1) separate visual and auditory lexicons and (2) modality dependence of implicit memory. In five experiments, we employed manipulations aimed to minimize study-test asymmetries between the two priming conditions. Activities at visual and auditory study were matched, words were phonologically consistent, and study modality was manipulated between subjects. Equal magnitudes of inter- and intramodality priming were found in experiments with visual and auditory stem completion at test, with visual fragment completion at test, and with visual and auditory perceptual identification at test. A within-subjects experiment yielded the conventional intramodality advantage. The results point to a single amodal lexicon and to modality-independent phonological processing as the basis of implicit word memory.     

跨视听通道的相继错误记忆效应

本研究以12名大学生为被试采用DRM范式,考察了视觉与听觉通道在编码阶段的错误记忆ERP效应,以从更深层面认识错误记忆的内在加工机制。研究结果表明,在编码阶段,在300-500ms及500-700ms的时间窗视觉通道均存在相继错误记忆效应（DIM）,比听觉通道表现出较强的DIM效应,这不仅说明视觉通道与听觉通道在编码阶段具有不同的脑机制,而且表明DIM效应与学习项目的语义加工有着密切的关系。     

Evidence for auditory temporal distinctiveness: modality effects in order and frequency judgments

Two new, long-lasting phenomena involving modality of stimulus presentation are documented. In one series of experiments we investigated effects of modality of presentation on order judgments. Order judgments for auditory words were more accurate than order judgments for visual words at both the beginning and the end of lists, and the auditory advantage increased with the temporal separation of the successive items. A second series of experiments investigated effects of modality on estimates of presentation frequency. Frequency estimates of repeated auditory words exceeded frequency estimates of repeated visual words. The auditory advantage increased with frequency of presentation, and this advantage was not affected by the retention interval. These various effects were taken as support for a temporal coding assumption, that auditory presentation produces a more accurate encoding of time of presentation than does visual presentation.     

Individual differences in vulnerability to subjective time distortion1

Time duration is perceived to be longer when accompanied by dynamic sensory stimulation than when accompanied by static stimulation. This distortion of time perception is thought to be due to the acceleration of an internal pacemaker that has been assumed to be the main component of temporal judgments. In order to investigate whether the function of the internal pacemaker is modality dependent or independent, we examined the correlation of visual flicker and auditory flutter effects on a temporal production task. While seeing a 10‐Hz visual flicker or hearing a 10‐Hz auditory flutter, participants estimated a duration of 2500 ms as accurately as possible by pressing a button. The results showed a significant within‐individual correlation between the time distortion due to visual flicker and that due to auditory flutter. Additionally, we found that time distortion due to auditory flutter tended to be larger in female participants than in male participants. These results suggest that the mechanisms underlying subjective time dilation are similar between vision and audition within individuals, but that they vary across individuals.