Dynamic adjustment of temporal preparation: Shifting warning signal modality attenuates the sequential foreperiod effect

We examined sequential effects in the variable foreperiod (FP) paradigm, which refer to the finding that responses to an imperative signal (IS) are fast when a short FP trial is repeated but slow when it is preceded by a long FP trial. The effect has been attributed to a trace-conditioning mechanism in which individuals learn the temporal relationship between a warning signal (WS) and the IS in a trial-by-trial manner. An important assumption is that the WS in a current trial (i.e., trial FP_n) acts as a conditioned stimulus, such that it automatically triggers the conditioned response at the exact critical moment that was imperative in the previous trial (i.e., trial FP_n−1). According to this assumption, a shift from one WS modality in trial FP_n−1 to another modality in trial FP_n is expected to eliminate or at least reduce the sequential FP effect. This prediction was tested in three experiments that included a random variation of WS modality and FP length within blocks of trials. In agreement with the prediction, a shift in WS modality attenuated the asymmetry of the sequential FP effect.     

Sequential effects within a short foreperiod context: evidence for the conditioning account of temporal preparation

Responses to an imperative stimulus (IS) are especially fast when they are preceded by a warning signal (WS). When the interval between WS and IS (the foreperiod, FP) is variable, reaction time (RT) is not only influenced by the current FP but also by the FP of the preceding trial. These sequential effects have recently been proposed to originate from a trace conditioning process, in which the individuals learn the temporal WS-IS relationship in a trial-by-trial manner. Research has shown that trace conditioning is maximal when the temporal interval between the conditioned and unconditioned stimulus is between 0.25 and 0.60 s. Consequently, one would predict that sequential effects occur especially within short FP contexts. However, this prediction is contradicted by Karlin [Karlin, L. (1959). Reaction time as a function of foreperiod duration and variability. Journal of Experimental Psychology, 58, 185-191] who did not observe the typical sequential effects with short FPs. To investigate temporal preparation for short FPs, three experiments were conducted, examining the sequential FP effect comparably for short and long FP-sets (Experiment 1), assessing the influence of catch trials (Experiment 2) and the case of a very dense FP-range (Experiment 3) on sequential FP effects. The results provide strong evidence for sequential effects within a short FP context and thus support the trace conditioning account of temporal preparation.     

The relation between physical stimulus properties and the effect of foreperiod duration on reaction time

Seven subjects were used in an experiment on the relation between signal modality and the effect of foreperiod duration (EP) on RT. With visual signals the usually reported systematic increase of RT as a function of FP duration (1, 5 and 15 s) was confirmed; with auditory signals no difference was found between FP's of 1 and 5 s while the effect at 15 s was equivalent to that found at 5 s with the visual signal. The results suggest that besides factors such as time uncertainty the FP effect is also largely dependent on the arousing quality of the signal.     

The effect of exogenous spatial attention on auditory information processing

This study investigated the effect of exogenous spatial attention on auditory information processing. In Experiments 1, 2 and 3, temporal order judgment tasks were performed to examine the effect. In Experiment 1 and 2, a cue tone was presented to either the left or right ear, followed by sequential presentation of two target tones. The subjects judged the order of presentation of the target tones. The results showed that subjects heard both tones simultaneously when the target tone, which was presented on the same side as the cue tone, was presented after the target tone on the opposite side. This indicates that spatial exogenous attention was aroused by the cue tone, and facilitated subsequent auditory information processing. Experiment 3 examined whether both cue position and frequency influence the resulting information processing. The same effect of spatial attention was observed, but the effect of attention to a certain frequency was only partially observed. In Experiment 4, a tone fusion judgment task was performed to examine whether the effect of spatial attention occurred in the initial stages of hearing. The result suggests that the effect occurred in the later stages of hearing.     

Arousal modulates temporal preparation under increased time uncertainty: Evidence from higher-order sequential foreperiod effects

When the foreperiod (FP) is unpredictably varied in reaction-time tasks, responses are slow at short but fast at long FPs (variable-FP effect), and further vary asymmetrically as a function of FP sequence (sequential FP effect). A trace-conditioning model attributes these phenomena to time-related associative learning, while a dual-process model views them as resulting from combined effects of strategic preparation and trial-to-trial changes in arousal. Sometimes, responses are slower in long-long than in short-long FP sequences. This pattern is not predicted from the trace-conditioning account, since FP repetitions should speed up, rather than slow down, responses (due to reinforcement). The effect, however, might indicate the contribution of arousal, which according to the dual-process model, is heightened after a short FP(n-1) but decreased after a long FP(n-1). In five experiments, we examined higher-order sequential FP effects on performance, with a particular emphasis on analyzing performance in long-FP(n) trials as a function of FP length in the two preceding trials, varying temporal FP context (i.e. average FP length) and reaction mode (simple vs. choice reaction). Slower responses in long-long-long (compared with short-short-long) FP sequences were not found within a short-FP context (Exps. 1 & 2) but clearly emerged within a long-FP context (Exps. 3-5). This pattern supports the notion that transient arousal changes contribute to sequential performance effects in variable-FP tasks, in line with the dual-process account of temporal preparation.     

选择性注意和分配性注意对多感觉整合的不同影响*

以简单图形为视觉刺激,以短纯音作为听觉刺激,通过指导被试注意不同通道（注意视觉、注意听觉、注意视听）以形成不同注意状态（选择性注意和分配性注意）,考察了注意对多感觉整合的影响,发现只有在分配性注意时被试对双通道目标的反应最快最准确。通过竞争模型分析发现,这种对双通道目标的加工优势源自于视听双通道刺激的整合。上述结果表明,只有在分配性注意状态下才会产生多感觉整合。     

The modality shift effect and the effectiveness of warning signals in different modalities

Which is better, a visual or an auditory warning signal? Initial findings suggested that an auditory signal was more effective, speeding reaction to a target more than a visual warning signal, particularly at brief foreperiods [Bertelson, P., & Tisseyre, F. (1969). The time-course of preparation: confirmatory results with visual and auditory warning signals. Acta Psychologica, 30. In W.G. Koster (Ed.), Attention and Performance II (pp. 145-154); Davis, R., & Green, F. A. (1969). Intersensory differences in the effect of warning signals on reaction time. Acta Psychologica, 30. In W.G. Koster (Ed.), Attention and Performance II (pp. 155-167)]. This led to the hypothesis that an auditory signal is more alerting than a visual warning signal [Sanders, A. F. (1975). The foreperiod effect revisited. Quarterly Journal of Experimental Psychology, 27, 591-598; Posner, M. I., Nissen, M. J., & Klein, R. M. (1976). Visual dominance: an information-processing account of its origins and significance. Psychological Review, 83, 157-171]. Recently [Turatto, M., Benso, F., Galfano, G., & Umilta, C. (2002). Nonspatial attentional shifts between audition and vision. Journal of Experimental Psychology: Human Perception and Performance, 28, 628-639] found no evidence for an auditory warning signal advantage and showed that at brief foreperiods a signal in the same modality as the target facilitated responding more than a signal in a different modality. They accounted for this result in terms of the modality shift effect, with the signal exogenously recruiting attention to its modality, and thereby facilitating responding to targets arriving in the modality to which attention had been recruited. The present study conducted six experiments to understand the cause of these conflicting findings. The results suggest that an auditory warning signal is not more effective than a visual warning signal. Previous reports of an auditory superiority appear to have been caused by using different locations for the visual warning signal and visual target, resulting in the target arriving at an unattended location when the foreperiod was brief. Turatto et al.'s results were replicated with a modality shift effect at brief foreperiods. However, it is also suggested that previous measures of the modality shift effect may still have been confounded by a location cuing effect.     

Effects of laterality and pitch height of an auditory accessory stimulus on horizontal response selection: The Simon effect and the SMARC effect

In the present article, we investigated the effects of pitch height and the presented ear (laterality) of an auditory stimulus, irrelevant to the ongoing visual task, on horizontal response selection. Performance was better when the response and the stimulated ear spatially corresponded (Simon effect), and when the spatial—musical association of response codes (SMARC) correspondence was maintained—that is, right (left) response with a high-pitched (low-pitched) tone. These findings reveal an automatic activation of spatially and musically associated responses by task-irrelevant auditory accessory stimuli. Pitch height is strong enough to influence the horizontal responses despite modality differences with task target.     

Detection and recognition of intensity changes in tone and noise: The detection-recognition disparity

Recognition of increments vs decrements in auditory intensity improves with signal duration, relative to detection of increments and decrements. This effect obtains whether the background stimulus is a tone, noise, or a tone with noise masker, and is largely uninfluenced by the rise time of the signals. These data are inconsistent with detection models in which the O makes only one sensory observation during each observation interval, but can be described in terms of a neural timing model in which transients play a critical role.     

Comparisons of memory for nonverbal auditory and visual sequential stimuli

Properties of auditory and visual sensory memory were compared by examining subjects' recognition performance of randomly generated binary auditory sequential frequency patterns and binary visual sequential color patterns within a forced-choice paradigm. Experiment 1 demonstrated serial-position effects in auditory and visual modalities consisting of both primacy and recency effects. Experiment 2 found that retention of auditory and visual information was remarkably similar when assessed across a 10 s interval. Experiments 3 and 4, taken together, showed that the recency effect in sensory memory is affected more by the type of response required (recognition vs. reproduction) than by the sensory modality employed. These studies suggest that auditory and visual sensory memory stores for nonverbal stimuli share similar properties with respect to serial-position effects and persistence over time.     

Sound enhances visual perception: cross-modal effects of auditory organization on vision

Six experiments demonstrated cross-modal influences from the auditory modality on the visual modality at an early level of perceptual organization. Participants had to detect a visual target in a rapidly changing sequence of visual distractors. A high tone embedded in a sequence of low tones improved detection of a synchronously presented visual target (Experiment 1), but the effect disappeared when the high tone was presented before the target (Experiment 2). Rhythmically based or order-based anticipation was unlikely to account for the effect because the improvement was unaffected by whether there was jitter (Experiment 3) or a random number of distractors between successive targets (Experiment 4). The facilitatory effect was greatly reduced when the tone was less abrupt and part of a melody (Experiments 5 and 6). These results show that perceptual organization in the auditory modality can have an effect on perceptibility in the visual modality.     

Shared attentional resources for processing visual and chemosensory information

For many years, researchers have argued that we have separate attentional resources for the processing of information impinging on each of our sensory receptor systems. However, a number of recent studies have demonstrated the existence of shared attentional resources for the processing of auditory, visual and tactile stimuli. In the present study, we examined whether there are also common attentional resources for the processing of chemosensory stimuli. Participants made speeded (left vs. right) footpedal discrimination responses to an unpredictable sequence of visual and chemosensory stimuli presented to either nostril. The participants' attention was directed to one or the other modality by means of a symbolic auditory cue (high or low tone) at the start of each trial, which predicted the likely modality for the upcoming target on the majority (80%) of trials. Participants responded more rapidly when the target occurred in the expected modality than when it occurred in the unexpected modality, implying the existence of shared attentional resources for the processing of chemosensory and visual stimuli.     

Categorization by schema relations and perceptual similarity in 5-year-olds and adults: A study in vision and in audition

This research explores the way in which young children (5 years of age) and adults use perceptual and conceptual cues for categorizing objects processed by vision or by audition. Three experiments were carried out using forced-choice categorization tasks that allowed responses based on taxonomic relations (e.g., vehicles) or on schema category relations (e.g., vehicles that can be seen on the road). In Experiment 1 (visual modality), prominent responses based on conceptually close objects (e.g., objects included in a schema category) were observed. These responses were also favored when within-category objects were perceptually similar. In Experiment 2 (auditory modality), schema category responses depended on age and were influenced by both within- and between-category perceptual similarity relations. Experiment 3 examined whether these results could be explained in terms of sensory modality specializations or rather in terms of information processing constraints (sequential vs. simultaneous processing).     

Sequential effects in rudimentary auditory and visual tasks

Three experiments examined sequential effects in choice reaction time tasks. On each trial, a right/left positional judgment was made to a either a pure tone or a luminance increment in a visual array of box elements. In the first two experiments, a preparatory signal was presented prior to each imperative signal to indicate the relevant stimulus modality. At a short stimulus onset asynchrony (SOA) between the preparatory and the imperative signal (i.e., 60 msec), subjects were quicker to repeat the same response than to change their response when presented with successive tones, although no such repetition effect occurred on the visual target trials. Subjects were impaired if the stimulus modality changed across successive trials regardless of the modality of the target. At a longer SOA (i.e., 500 msec), these sequential effects were abolished; subjects were assumed to be able to prepare for the relevant modality because of the presentation of the preparatory signal. When the preparatory signals were omitted, in a final experiment, the modality-switching costs were still evident, but now inhibition of return occurred on both the auditory and the visual target trials-subjects were now impaired in responding when the target reappeared at its immediately previous location. It seems, therefore, that the repetition effect and modality-switching effects do dissociate. The data revealed clear differences between orienting attention to a particular spatial locale and focusing attention to a particular sensory modality.     

Sustained effect of auditory entrainment on sequential tapping: The role of movement path complexity

The effects of auditory-motor entrainment have generally been investigated with periodic movements. Previous research has focused on how auditory-motor entrainment is influenced by the temporal structure of rhythms. The present study aimed to investigate whether auditory entrainment improved timing performance of sequential movements with varied path structures, and whether path complexity would affect any possible sustained effect of auditory entrainment. We also investigated whether the sustained effect was moderated by hearing single- vs. multiple-pitch audio prompts. Thirty participants were enrolled to perform a sequential finger-tapping task with discrete targets, in which the algebraic ratio relation of path lengths was manipulated as path complexity. Participants completed three stages per trial: initiation (to introduce the path sequence), entrainment (tapping along with the auditory and visual cues), and timekeeping (repeating the sequence without cues). We found timing improvement in terms of mean asynchronies and absolute interval error decrease after auditory entrainment. Only interval accuracy performance during timekeeping and entrainment was affected by path complexity. Moreover, no clear difference was observed between the rhythm sets in terms of single vs. multiple pitches. In conclusion, we found that phase and interval duration accuracy of predefined isochronous sequential movements with varied path complexity can be improved by auditory entrainment, and that auditory entrainment affects our performance beyond the actual presence of the auditory cue.     

The perceptual basis of the modality effect in multimedia learning

Various studies have demonstrated an advantage of auditory over visual text modality when learning with texts and pictures. To explain this modality effect, two complementary assumptions are proposed by cognitive theories of multimedia learning: first, the visuospatial load hypothesis, which explains the modality effect in terms of visuospatial working memory overload in the visual text condition; and second, the temporal contiguity assumption, according to which the modality effect occurs because solely auditory texts and pictures can be attended to simultaneously. The latter explanation applies only to simultaneous presentation, the former to both simultaneous and sequential presentation. This paper introduces a third explanation, according to which parts of the modality effect are due to early, sensory processes. This account predicts that-for texts longer than one sentence-the modality effect with sequential presentation is restricted to the information presented most recently. Two multimedia experiments tested the influence of text modality across three different conditions: simultaneous presentation of texts and pictures versus sequential presentation versus presentation of text only. Text comprehension and picture recognition served as dependent variables. An advantage for auditory texts was restricted to the most recent text information and occurred under all presentation conditions. With picture recognition, the modality effect was restricted to the simultaneous condition. These findings clearly support the idea that the modality effect can be attributed to early processes in perception and sensory memory rather than to a working memory bottleneck.     

The relationship between reaction time and intensity in discrete auditory tasks

The effect of signal intensity upon reaction time (RT) was studied in three auditory RT tasks in which the signal was a tone of high or low frequency. Experiment I showed the well-known negative gradient with intensity of simple RT when the subject was instructed to ignore the frequency and give the same response to both tones. But when the subject had to discriminate the frequency in a choice RT task, the RT/intensity relationship appeared to be U-shaped. Experiment II showed that when the subject was required to make a response to one signal but withhold it for the other, a task which requires discrimination of the frequency of the tone but removes the necessity to choose between overt responses, no increase in RT at high intensities was obtained. The results indicate that it is the response choice stage rather than the stimulus encoding stage which is retarded at higher energy levels. Experiment I also demonstrated that visual and auditory leading signals have similar facilitating effects without affecting the RT/intensity relationships.     

Directing spatial attention towards the illusory location of a ventriloquized sound

In this study, we examined whether ventriloquism can rearrange external space on which spatial reflexive attention operates. The task was to judge the elevation (up vs down) of auditory targets delivered in the left or the right periphery, taking no account of side of presentation. Targets were preceded by either auditory, visual, or audiovisual cues to that side. Auditory, but not visual cues had an effect on the speed of auditory target discrimination. On the other hand, a ventriloquized cue, consisting of a tone in central location synchronized with a light flash in the periphery, facilitated responses to targets appearing on the same side as the flash. That effect presumably resulted from the attraction of the apparent location of the tone towards the flash, a well-known manifestation of ventriloquism. Ventriloquism thus can reorganize space in which reflexive attention operates.     

Attentional capture in serial audiovisual search tasks

The phenomenon of attentional capture has typically been studied in spatial search tasks. Dalton and Lavie recently demonstrated that auditory attention can also be captured by a singleton item in a rapidly presented tone sequence. In the experiments reported here, we investigated whether these findings extend cross-modally to sequential search tasks using audiovisual stimuli. Participants searched a stream of centrally presented audiovisual stimuli for targets defined on a particular dimension (e.g., duration) in a particular modality. Task performance was compared in the presence versus absence of a unique singleton distractor. Irrelevant auditory singletons captured attention during visual search tasks, leading to interference when they coincided with distractors but to facilitation when they coincided with targets. These results demonstrate attentional capture by auditory singletons during nonspatial visual search.     

听觉通道下负数的低水平加工引起注意的SNARC效应

采用刺激探测任务,探讨了听觉通道下负数的低水平加工能否引起注意的SNARC效应。结果表明:(1)负数的低水平加工能够产生注意的SNARC效应,即线索为绝对值小的负数时,被试对左耳目标反应更快;线索为绝对值大的负数时,被试对右耳目标反应更快。(2)负数的加工所引起听觉空间注意的转移并不依赖于负数所在的情境,而仅通过其绝对值大小与空间表征发生关联。这一结论与视觉条件下所得到的结论不一致,并支持了系统发生论假说。