Influence of auditory and audiovisual stimuli on the right–left prevalence effect

When auditory stimuli are used in two-dimensional spatial compatibility tasks, where the stimulus and response configurations vary along the horizontal and vertical dimensions simultaneously, a right–left prevalence effect occurs in which horizontal compatibility dominates over vertical compatibility. The right–left prevalence effects obtained with auditory stimuli are typically larger than that obtained with visual stimuli even though less attention should be demanded from the horizontal dimension in auditory processing. In the present study, we examined whether auditory or visual dominance occurs when the two-dimensional stimuli are audiovisual, as well as whether there will be cross-modal facilitation of response selection for the horizontal and vertical dimensions. We also examined whether there is an additional benefit of adding a pitch dimension to the auditory stimulus to facilitate vertical coding through use of the spatial-musical association of response codes (SMARC) effect, where pitch is coded in terms of height in space. In Experiment 1, we found a larger right–left prevalence effect for unimodal auditory than visual stimuli. Neutral, non-pitch coded, audiovisual stimuli did not result in cross-modal facilitation, but did show evidence of visual dominance. The right–left prevalence effect was eliminated in the presence of SMARC audiovisual stimuli, but the effect influenced horizontal rather than vertical coding. Experiment 2 showed that the influence of the pitch dimension was not in terms of influencing response selection on a trial-to-trial basis, but in terms of altering the salience of the task environment. Taken together, these findings indicate that in the absence of salient vertical cues, auditory and audiovisual stimuli tend to be coded along the horizontal dimension and vision tends to dominate audition in this two-dimensional spatial stimulus–response task.     

Assessing automaticity in the audiovisual integration of motion

This study examined the multisensory integration of visual and auditory motion information using a methodology designed to single out perceptual integration processes from post-perceptual influences. We assessed the threshold stimulus onset asynchrony (SOA) at which the relative directions (same vs. different) of simultaneously presented visual and auditory apparent motion streams could no longer be discriminated (Experiment 1). This threshold was higher than the upper threshold for direction discrimination (left vs. right) of each individual modality when presented in isolation (Experiment 2). The poorer performance observed in bimodal displays was interpreted as a consequence of automatic multisensory integration of motion information. Experiment 3 supported this interpretation by ruling out task differences as the explanation for the higher threshold in Experiment 1. Together these data provide empirical support for the view that multisensory integration of motion signals can occur at a perceptual level.     

From auditory image to auditory percept: Facilitation through common processes?

Two experiments explored implicit memory for auditory stimuli as measured by a test of perceptual identification. The facilitative effect of perceived auditory primes was contrasted with that of imaged auditory primes. In Experiment 1, there was a significant priming effect from imaged spoken-word primes that did not differ significantly from the level of priming due to perceived spoken-word primes, measured by a test of auditory perceptual identification. There was no facilitation of spoken-word identification following creation of an image of a word’s referent sound. In Experiment 2, identification of an environmental sound was facilitated by prior processing of an imaged sound from the same category, though there was significantly more transfer following processing of the actual sound.     

How does reading direction modulate perceptual asymmetry effects?

Left-side bias effects refer to a bias towards the left side of the stimulus/space in perceptual/visuospatial judgments, and are argued to reflect dominance of right hemisphere processing. It remains unclear whether reading direction can also account for the bias effect. Previous studies comparing readers of languages read from left to right with those read from right to left (e.g., French vs. Hebrew) have obtained inconsistent results. As a language that can be read from left to right or from right to left, Chinese provides a unique opportunity for a within-culture examination of reading direction effects. Chinese participants performed a perceptual judgment task (with both face and Chinese character stimuli; Experiment 1) and two visuospatial attention tasks (the greyscales and line bisection tasks; Experiment 2) once before and once after a reading task, in which they read Chinese passages either from left to right or from right to left for about 20 min. After reading from right to left, participants showed significantly reduced left-side bias in Chinese character perceptual judgments but not in the other three tasks. This effect suggests that the role of reading direction on different forms of left-side bias may differ, and its modulation may be stimulus-specific.     

The role of preparation and cue-modality in crossmodal task switching

The present study examined the effects of cue-based preparation and cue-target modality mapping in crossmodal task switching. In two experiments, we randomly presented lateralized visual and auditory stimuli simultaneously. Subjects were asked to make a left/right judgment for a stimulus in only one of the modalities. Prior to each trial, the relevant stimulus modality was indicated by a visual or auditory cue. The cueing interval was manipulated to examine preparation. In Experiment 1, we used a corresponding mapping of cue-modality and stimulus modality, whereas in Experiment 2 the mapping of cue and stimulus modalities was reversed. We found reduced modality-switch costs with a long cueing interval, showing that attention shifts to stimulus modalities can be prepared, irrespective of cue-target modality mapping. We conclude that perceptual processing in crossmodal switching can be biased in a preparatory way towards task-relevant stimulus modalities.     

The effect of exogenous spatial attention on auditory information processing

This study investigated the effect of exogenous spatial attention on auditory information processing. In Experiments 1, 2 and 3, temporal order judgment tasks were performed to examine the effect. In Experiment 1 and 2, a cue tone was presented to either the left or right ear, followed by sequential presentation of two target tones. The subjects judged the order of presentation of the target tones. The results showed that subjects heard both tones simultaneously when the target tone, which was presented on the same side as the cue tone, was presented after the target tone on the opposite side. This indicates that spatial exogenous attention was aroused by the cue tone, and facilitated subsequent auditory information processing. Experiment 3 examined whether both cue position and frequency influence the resulting information processing. The same effect of spatial attention was observed, but the effect of attention to a certain frequency was only partially observed. In Experiment 4, a tone fusion judgment task was performed to examine whether the effect of spatial attention occurred in the initial stages of hearing. The result suggests that the effect occurred in the later stages of hearing.     

Visual and auditory accessory stimulus offset and the Simon effect

We investigated the effect on the right and left responses of the disappearance of a task-irrelevant stimulus located on the right or left side. Participants pressed a right or left response key on the basis of the color of a centrally located visual target. Visual (Experiment 1) or auditory (Experiment 2) task-irrelevant accessory stimuli appeared or disappeared at locations to the right or left of the central target. In Experiment 1, responses were faster when onset or offset of the visual accessory stimulus was spatially congruent with the response. In Experiment 2, responses were again faster when onset of the auditory accessory stimulus and the response were on the same side. However, responses were slightly slower when offset of the auditory accessory stimulus and the response were on the same side than when they were on opposite sides. These findings indicate that transient change information is crucial for a visual Simon effect, whereas sustained stimulation from an ongoing stimulus also contributes to an auditory Simon effect.     

Auditory/visual duration bisection in patients with left or right medial-temporal lobe resection

Patients with unilateral (left or right) medial temporal lobe lesions and normal control (NC) volunteers participated in two experiments, both using a duration bisection procedure. Experiment 1 assessed discrimination of auditory and visual signal durations ranging from 2 to 8 s, in the same test session. Patients and NC participants judged auditory signals as longer than equivalent duration visual signals. The difference between auditory and visual time discrimination was equivalent for the three groups, suggesting that a unilateral temporal lobe resection does not modulate the modality effect. To document interval-timing abilities after temporal lobe resection for different duration ranges, Experiment 2 investigated the discrimination of brief, 50-200 ms, auditory durations in the same patients. Overall, patients with right temporal lobe resection were found to have more variable duration judgments across both signal modality and duration range. These findings suggest the involvement of the right temporal lobe at the level of the decision process in temporal discriminations.     

Irrelevant auditory attention shifts prime corresponding responses

In this paper we investigate whether an attention shift towards an auditory signal, while performing a two-choice serial reaction time task, primes responses in the direction of the auditory signal. In Experiment 1, subjects had to react to the pitch of the signal, which was randomly presented to the left or right ear. A short (50-ms) and a long (1000-ms) response-stimulus interval (RSI) was used. In Experiment 2 with an RSI of 2000 ms, subjects had to respond to the color of a centrally presented visual stimulus, while a sound was presented to one of the ears. In the short RSI condition of Experiment 1, there was a Simon effect for location alternations only. In the long RSI condition, there was a Simon effect for location repetitions and alternations. Experiment 2 showed a Simon effect in relation to the accessory sound. The results of this study suggest that an attention shift is a sufficient and necessary condition in order to observe a Simon effect. Preparation of this article was supported by a grant from the National Fund for Scientific Research, Flanders, Belgium (FWO-VL G.0009.99)     

Material-specific perceptual interference for visual words and faces: implications for models of capacity limitations, attention, and laterality

Four experiments were conducted to test the hypothesis that perceptual interference effects would be greatest among items that are processed by the same hemisphere than among items processed by different hemispheres. Recognition of two peripheral faces, flanked on either side of a central item that also had to be identified, was worse when the item was a face than when the item was a word. The opposite results were obtained for peripheral word identification by naming (Experiment 1). Identical results were obtained when a nonsense form was substituted for the central face (Experiment 2). Since unfamiliar faces and nonsense forms are processed primarily by the right hemisphere and words are processed by the left, the results are consistent with the hypothesis. The interference effect was the same even when the central nonsense shape was identified by naming, suggesting that the locus of the effect is not at response selection (Experiment 3). Ignoring the central item only reduces the interference effects but does not eliminate it (Experiment 4). Although in general words were perceived better in the right visual field, and faces were perceived better in the left, small fluctuations in the magnitude but not in the direction of perceptual asymmetries were sometimes noted with changes in the central item. The fluctuations were usually opposite to those predicted by attentional models of perceptual asymmetries.     

On the locus of the semantic satiation effect: Evidence from event-related brain potentials

The present study sought to determine whether semantic satiation is merely a by-product of adaptation or satiation of upstream, nonsemantic perceptual processes or whether the effect can have a locus in semantic memory. This was done by measuring event-related brain potentials (ERPs) in a semantic word-detection task involving multiple presentations of primes and critical related and unrelated words in three experiments involving visual (Experiment 1) and auditory (Experiments 2A and 2B) stimuli. Primes varied in their type case (Experiment 1) or pitch (Experiment 2B) in order to discourage sensory adaptation. Prime satiation and relatedness of the primes to the critical word had interacting effects on ERP amplitude to critical words, particularly within the time-window of the N400 component. Because numerous studies have indicated a role for the N400 in semantic processing, modulation of the N400 relatedness effect by prime satiation (with little or no contribution from perceptual adaptation) suggests that semantic memory can be directly satiated, rather than the cost to semantic processing necessarily resulting from impoverishment of perceptual inputs.     

Perceptual and decisional contributions to audiovisual interactions in the perception of apparent motion: a signal detection study

Motion information available to different sensory modalities can interact at both perceptual and post-perceptual (i.e., decisional) stages of processing. However, to date, researchers have only been able to demonstrate the influence of one of these components at any given time, hence the relationship between them remains uncertain. We addressed the interplay between the perceptual and post-perceptual components of information processing by assessing their influence on performance within the same experimental paradigm. We used signal detection theory to discriminate changes in perceptual sensitivity (d') from shifts in response criterion (c) in performance on a detection (Experiment 1) and a classification (Experiment 2) task regarding the direction of auditory apparent motion streams presented in noise. In the critical conditions, a visual motion distractor moving either leftward or rightward was presented together with the auditory motion. The results demonstrated a significant decrease in sensitivity to the direction of the auditory targets in the crossmodal conditions as compared to the unimodal baseline conditions that was independent of the relative direction of the visual distractor. In addition, we also observed significant shifts in response criterion, which were dependent on the relative direction of the distractor apparent motion. These results support the view that the perceptual and decisional components involved in audiovisual interactions in motion processing can coexist but are largely independent of one another.     

Unilateral hemispheric activation does not affect free-viewing perceptual asymmetries

Strong leftward perceptual biases have been reported for the selection of the darker of two left/right mirror-reversed luminance gradients under free-viewing conditions. This study investigated the effect of unilateral hemispheric activation on this leftward bias in two groups of dextrals (N = 52 and N = 24). In Experiment 1, activation was manipulated by asking participants to tap with their left or right fingers along their midline. In Experiment 2, participants clenched their left or right hands in their respective hemispaces. Participants selected the stimulus that was darker on the left-hand side 73% of the time. Despite manipulations of activation strength and hemispace, activation had no effect on the asymmetry. If activation was important, the leftward bias should have been enhanced when the left hand/right hemisphere was active and reduced (or reversed) when the right hand/left hemisphere was active. The contribution of left-to-right scanning biases to free-viewing perceptual asymmetries is discussed as an alternative.     

The inability to ignore auditory distractors as a function of visual task perceptual load

Using a response competition paradigm, we investigated the ability to ignore target response-compatible, target response-incompatible, and neutral visual and auditory distractors presented during a visual search task. The perceptual load model of attention (e.g., Lavie & Tsal, 1994) states that task-relevant processing load determines irrelevant distractor processing in such a way that increasing processing load prevents distractor processing. In three experiments, participants searched sets of one (easy search) or six (hard search) similar items. In Experiment 1, visual distractors influenced reaction time (RT) and accuracy only for easy searches, following the perceptual load model. Surprisingly, auditory distractors yielded larger distractor compatibility effects (median RT for incompatible trials minus median RT for compatible trials) for hard searches than for easy searches. In Experiments 2 and 3, during hard searches, consistent RT benefits with response-compatible and RT costs with response-incompatible auditory distractors occurred only for hard searches. We suggest that auditory distractors are processed regardless of visual perceptual load but that the ability to inhibit cross-modal influence from auditory distractors is reduced under high visual load.     

Knowing when to hear aids what to hear

Temporal preparation often has been assumed to influence motor stages of information processing. Recent studies, however, challenge this notion and provide evidence for a facilitation of visual processing. The present study was designed to investigate whether perceptual processing in the auditory domain also benefits from temporal preparation. To this end, we employed a pitch discrimination task. In Experiment 1, discrimination performance was clearly improved when participants were temporally prepared. This finding was confirmed in Experiment 2, which ruled out possible influences of short-term memory. The results support the notion that temporal preparation enhances perceptual processing not only in the visual, but also in the auditory, modality.     

Knowing when to hear aids what to hear

Temporal preparation often has been assumed to influence motor stages of information processing. Recent studies, however, challenge this notion and provide evidence for a facilitation of visual processing. The present study was designed to investigate whether perceptual processing in the auditory domain also benefits from temporal preparation. To this end, we employed a pitch discrimination task. In Experiment 1, discrimination performance was clearly improved when participants were temporally prepared. This finding was confirmed in Experiment 2, which ruled out possible influences of short-term memory. The results support the notion that temporal preparation enhances perceptual processing not only in the visual, but also in the auditory, modality.     

Orientation asymmetry in the flanker task

Experiments 1 and 2 of this study show that when the target is either a vertical or a horizontal line, diagonal-line flankers tilted 45° either to the right or to the left have the same effect as do incongruent flankers. When the target is a diagonal line tilted either to the right or to the left, vertical- or horizontal-line flankers do not have the same effect as do incongruent flankers. Experiment 3 demonstrates that this asymmetry is not caused by the temporal-dynamic aspects of the processing. Together, these experiments suggest that there is an asymmetrical relation between diagonal lines and either vertical or horizontal lines otttside of the central focus of attention. Experiment 4 shows that despite this asymmetry in the flanker task, visual search for a vertical- or a horizontal-line target among diagonal-line distractors is not affected by the number of distractors. Possible explanations of this phenomenon are discussed.     

Lexical influences in audiovisual speech perception

Phoneme identification with audiovisually discrepant stimuli is influenced hy information in the visual signal (the McGurk effect). Additionally, lexical status affects identification of auditorily presented phonemes. The present study tested for lexical influences on the McGurk effect. Participants identified phonemes in audiovisually discrepant stimuli in which lexical status of the auditory component and of a visually influenced percept was independently varied. Visually influenced (McGurk) responses were more frequent when they formed a word and when the auditory signal was a nonword (Experiment 1). Lexical effects were larger for slow than for fast responses (Experiment 2), as with auditory speech, and were replicated with stimuli matched on physical properties (Experiment 3). These results are consistent with models in which lexical processing of speech is modality independent.     

Duration discrimination of filled and empty auditory intervals: Cognitive and perceptual factors

Adult subjects were presented with two auditory stimuli per trial, and their task was to decide which of the two was longer in duration. An adaptive psychophysical procedure was used. In Experiments 1, 2, and 4, the base duration was 50 msec, whereas in Experiment 3, the base duration was 1 sec. In Experiments 1, 2, and 4, it was found that filled intervals (continuous tones) were discriminated more accurately than empty intervals (with onset and offset marked by clicks). It was concluded that this difference was perceptual rather than cognitive in nature, since performance on filled and empty intervals was not affected by increasing cognitive load in a dual-task procedure (Experiment 2) but was affected by backward masking (Experiment 4). In contrast, the results of Experiment 3 showed that duration discrimination of filled auditory intervals of longer duration was cognitively influenced, since performance was impaired by increasing cognitive load. Implications for notions of perceptual processing and timing mechanisms tanderlying differences in duration discrimination with filled and empty intervals are discussed.     

Duration discrimination of filled and empty auditory intervals: cognitive and perceptual factors.

Adult subjects were presented with two auditory stimuli per trial, and their task was to decide which of the two was longer in duration. An adaptive psychophysical procedure was used. In Experiments 1, 2, and 4, the base duration was 50 msec, whereas in Experiment 3, the base duration was 1 sec. In Experiments 1, 2, and 4, it was found that filled intervals (continuous tones) were discriminated more accurately than empty intervals (with onset and offset marked by clicks). It was concluded that this difference was perceptual rather than cognitive in nature, since performance on filled and empty intervals was not affected by increasing cognitive load in a dual-task procedure (Experiment 2) but was affected by backward masking (Experiment 4). In contrast, the results of Experiment 3 showed that duration discrimination of filled auditory intervals of longer duration was cognitively influenced, since performance was impaired by increasing cognitive load. Implications for notions of perceptual processing and timing mechanism underlying differences in duration discrimination with filled and empty intervals are discussed.