Irrelevant speech effects and statistical learning

Immediate serial recall of visually presented verbal stimuli is impaired by the presence of irrelevant auditory background speech, the so-called irrelevant speech effect. Two of the three main accounts of this effect place restrictions on when it will be observed, limiting its occurrence either to items processed by the phonological loop (the phonological loop hypothesis) or to items that are not too dissimilar from the irrelevant speech (the feature model). A third, the object-oriented episodic record (O-OER) model, requires only that the memory task involves seriation. The present studies test these three accounts by examining whether irrelevant auditory speech will interfere with a task that does not involve the phonological loop, does not use stimuli that are compatible with those to be remembered, but does require seriation. Two experiments found that irrelevant speech led to lower levels of performance in a visual statistical learning task, offering more support for the O-OER model and posing a challenge for the other two accounts.     

Transfer of noncorresponding spatial associations to the auditory Simon task

Four experiments examined transfer of noncorresponding spatial stimulus-response associations to an auditory Simon task for which stimulus location was irrelevant. Experiment 1 established that, for a horizontal auditory Simon task, transfer of spatial associations occurs after 300 trials of practice with an incompatible mapping of auditory stimuli to keypress responses. Experiments 2-4 examined transfer effects within the auditory modality when the stimuli and responses were varied along vertical and horizontal dimensions. Transfer occurred when the stimuli and responses were arrayed along the same dimension in practice and transfer but not when they were arrayed along orthogonal dimensions. These findings indicate that prior task-defined associations have less influence on the auditory Simon effect than on the visual Simon effect, possibly because of the stronger tendency for an auditory stimulus to activate its corresponding response.     

The role of crossmodal competition and dimensional overlap in crossmodal attention switching

Crossmodal selective attention was investigated in a cued task switching paradigm using bimodal visual and auditory stimulation. A cue indicated the imperative modality. Three levels of spatial S–R associations were established following perceptual (location), structural (numerical), and conceptual (verbal) set-level compatibility. In Experiment 1, participants switched attention between the auditory and visual modality either with a spatial-location or spatial-numerical stimulus set. In the spatial-location set, participants performed a localization judgment on left vs. right presented stimuli, whereas the spatial-numerical set required a magnitude judgment about a visually or auditorily presented number word. Single-modality blocks with unimodal stimuli were included as a control condition. In Experiment 2, the spatial-numerical stimulus set was replaced by a spatial-verbal stimulus set using direction words (e.g., “left”). RT data showed modality switch costs, which were asymmetric across modalities in the spatial-numerical and spatial-verbal stimulus set (i.e., larger for auditory than for visual stimuli), and congruency effects, which were asymmetric primarily in the spatial-location stimulus set (i.e., larger for auditory than for visual stimuli). This pattern of effects suggests task-dependent visual dominance.     

Parallel scanning of auditory and visual information

Two experiments involving memory retrieval of auditorilv and visually presented materials were performed. In Experiment I, subjects were presented with memory sets of 1, 2, or 4 stimuli and then with a test item to be classified as belonging or not belonging to the memory set. In Condition 1, each memory stimulus was a single, auditorily presented letter. In Condition 2, each memory stimulus was a visually presented letter. In Conditions 3 and 4, each memory stimulus was a pair of letters, one presented visually and the other auditorily. Mean reaction time (RT) for the classification task increased as a function of number of memory stimuli at equal rates for all four conditions. This was interpreted as evidence for a parallel scanning process in Conditions 3 and 4 where the auditory item and visual item of each memory stimulus pair can be scanned simultaneously. Experiment II compared memory retrieval for a simultaneous condition in which auditory and visual memory items were presented as pairs with a sequential condition in which mixed auditory-visual memory sets were presented one item at a time. RTs were shorter for the simultaneous condition. This was interpreted as evidence that parallel scanning may depend upon memory input parameters.     

Generation of visual representations

The visual confusability of uppercase letters was manipulated in a successive same-different task to study the conditions under which visual generation from auditory inputs would occur and to investigate the figural specificity of the generated representations. Prior experiments have shown that visual confusions do occur when the initial stimulus is auditory and the second one is visual, which indicates that auditory stimuli can be encoded into visual forms. There has been some suggestion, however, that the generated visual code may have been too abstract to differentiate between the two cases in which letters can appear. In the present experiment, although the confusion effect was not eliminated when the subjects had no advance knowledge regarding the case in which the visual stimulus would appear, the marked confusion effect obtained when the visual stimulus was an uppercase letter was substantially attenuated when the letter appeared in lowercase. This was taken to indicate that the visual characteristics of a generated visual representation may be relatively specific. The results also suggested that subjects may wait until after the second stimulus is presented before they generate the visual representation of the initial auditory stimulus.     

Action effects in saccade control

According to the ideomotor principle, action preparation involves the activation of associations between actions and their effects. However, there is only sparse research on the role of action effects in saccade control. Here, participants responded to lateralized auditory stimuli with spatially compatible saccades toward peripheral targets (e.g., a rhombus in the left hemifield and a square in the right hemifield). Prior to the imperative auditory stimulus (e.g., a left tone), an irrelevant central visual stimulus was presented that was congruent (e.g., a rhombus), incongruent (e.g., a square), or unrelated (e.g., a circle) to the peripheral saccade target (i.e., the visual effect of the saccade). Saccade targets were present throughout a trial (Experiment 1) or appeared after saccade initiation (Experiment 2). Results showed shorter response times and fewer errors in congruent (vs. incongruent) conditions, suggesting that associations between oculomotor actions and their visual effects play an important role in saccade control.     

The inability to ignore auditory distractors as a function of visual task perceptual load

Using a response competition paradigm, we investigated the ability to ignore target response-compatible, target response-incompatible, and neutral visual and auditory distractors presented during a visual search task. The perceptual load model of attention (e.g., Lavie & Tsal, 1994) states that task-relevant processing load determines irrelevant distractor processing in such a way that increasing processing load prevents distractor processing. In three experiments, participants searched sets of one (easy search) or six (hard search) similar items. In Experiment 1, visual distractors influenced reaction time (RT) and accuracy only for easy searches, following the perceptual load model. Surprisingly, auditory distractors yielded larger distractor compatibility effects (median RT for incompatible trials minus median RT for compatible trials) for hard searches than for easy searches. In Experiments 2 and 3, during hard searches, consistent RT benefits with response-compatible and RT costs with response-incompatible auditory distractors occurred only for hard searches. We suggest that auditory distractors are processed regardless of visual perceptual load but that the ability to inhibit cross-modal influence from auditory distractors is reduced under high visual load.     

Stimulus suffixes and visual presentation

Seven experiments studied whether irrelevant visual stimuli (stimulus suffixes) would interfere with immediate serial recall of supraspan lists of digits presented visually. Across experiments a wide number of conditions were run, varying in method of presentation (sequential or simultaneous), rate of list presentation, and presence or absence of articulatory suppression. In no condition did a visual suffix have a significant detrimental effect on recall. These results stand in marked contrast to those found when auditory lists and suffixes have been used.     

Task switching: on the origin of response congruency effects

When people frequently alternate between simple cognitive tasks, performance on stimuli which are assigned the same response in both tasks is typically faster and more accurate than on stimuli which require different responses for both tasks, thus indicating stimulus processing according to the stimulus-response (S-R) rules of the currently irrelevant task. It is currently under debate whether such response congruency effects are mediated by the activation of an abstract representation of the irrelevant task in working memory or by "direct" associations between specific stimuli and responses. We contrasted these views by manipulating concurrent memory load (Experiment 1) and the frequency of specific S-R associations (Experiment 2). While between-task response congruency effects were not affected by the amount of concurrent memory load, they were much stronger for stimuli that were processed frequently in the context of a competitor task. These findings are consistent with the idea that a large portion of the congruency effects stems from direct S-R associations and they do not support a sole mediation by task-set activation in working memory.     

Selective impairment of auditory selective attention under concurrent cognitive load

Load theory predicts that concurrent cognitive load impairs selective attention. For visual stimuli, it has been shown that this impairment can be selective: Distraction was specifically increased when the stimulus material used in the cognitive load task matches that of the selective attention task. Here, we report four experiments that demonstrate such selective load effects for auditory selective attention. The effect of two different cognitive load tasks on two different auditory Stroop tasks was examined, and selective load effects were observed: Interference in a nonverbal-auditory Stroop task was increased under concurrent nonverbal-auditory cognitive load (compared with a no-load condition), but not under concurrent verbal-auditory cognitive load. By contrast, interference in a verbal-auditory Stroop task was increased under concurrent verbal-auditory cognitive load but not under nonverbal-auditory cognitive load. This double-dissociation pattern suggests the existence of different and separable verbal and nonverbal processing resources in the auditory domain.     

Mixing location-relevant and irrelevant tasks: Spatial compatibility effects eliminated by stimuli that share the same spatial codes

If compatible and incompatible mappings of left-right stimuli to left-right keypresses are mixed such that participants must maintain both task sets, the typical stimulus-response compatibility (SRC) effect is eliminated. Four experiments examined whether the SRC effect is influenced similarly by inclusion of trials on which stimulus location is irrelevant. The SRC effect for the location-relevant trials was eliminated in Experiments 1 and 3 when those trials shared left-right codes with the location-irrelevant trials. The SRC effect was not eliminated in Experiments 2 and 4, in which the location-relevant stimuli varied along the left-right dimension but the location-irrelevant stimuli were presented in the centre of the screen or in top-bottom locations. The Simon effect for the location-irrelevant trials in Experiments 1 and 3 was positive when the location-relevant mapping was compatible and reversed when it was incompatible. The results are generally consistent with an alternative-routes model, according to which a direct response-selection route is suppressed when compatible trials are mixed with location-irrelevant trials. Repetition analyses suggest that this suppression does not vary on a trial-to-trial basis.     

Effects of visual and auditory short-term memory tasks on the spatiotemporal dynamics and variability of postural sway

The authors measured postural sway while participants (N = 20 in each experiment) stood on a rigid or a compliant surface, with their eyes open or closed, and while they did or did not perform a short-term memory (STM) task. In Experiment 1, the STM stimuli were presented visually; in Experiment 2, the stimuli were presented auditorily. In both experiments, fine-scaled, mediolateral postural-sway variability decreased as the cognitive load imposed by the STM task increased. That effect was independent of support surface and vision manipulations. The spatiotemporal profile of postural sway was affected by both visual and auditory STM tasks, but to a greater degree by the auditory task. The authors discuss implications of the results for theories and models of postural control.     

The additivity of stimulus-response compatibility with perceptual and motor factors in a visual choice reaction time task

One hundred and four participants performed a two-choice reaction time task requiring button-press responses to visual stimuli. Two levels of stimulus-response compatibility were factorially combined with two levels of stimulus discriminability and two levels of response repertoire. Results showed that the effects of these three variables were additive both in terms of mean reaction time as in terms of reaction time variances. The implication of this outcome is discussed in terms of the underlying information processing stage structure.     

Behold the voice of wrath: cross-modal modulation of visual attention by anger prosody

Emotionally relevant stimuli are prioritized in human information processing. It has repeatedly been shown that selective spatial attention is modulated by the emotional content of a stimulus. Until now, studies investigating this phenomenon have only examined within-modality effects, most frequently using pictures of emotional stimuli to modulate visual attention. In this study, we used simultaneously presented utterances with emotional and neutral prosody as cues for a visually presented target in a cross-modal dot probe task. Response times towards targets were faster when they appeared at the location of the source of the emotional prosody. Our results show for the first time a cross-modal attentional modulation of visual attention by auditory affective prosody.     

Effects of auditory stimulus intensity on response force in simple, go/no-go, and choice RT tasks

In four experiments, increasing the intensities of both relevant and irrelevant auditory stimuli was found to increase response force (RF) in simple, go/no-go, and choice reaction time (RT) tasks. These results raise problems for models that localize the effects of auditory intensity on purely perceptual processes, indicating instead that intensity also affects motor output processes under many circumstances. In Experiment 1, simple RT, go/no-go, and choice RT tasks were compared, using the same stimuli for all tasks. Auditory stimulus intensity affected both RT and RF, and these effects were not modulated by task. In Experiments 2-4, an irrelevant auditory accessory stimulus accompanied a relevant visual stimulus, and the go/no-go and choice tasks were used. The intensity of the irrelevant auditory accessory stimulus was found to affect RT and RF, although the sizes of these effects depended somewhat on the temporal predictability of the accessory stimulus.     

Validity and boundary conditions of automatic response activation in the Simon task.

Three experiments were conducted to determine whether spatial stimulus-response compatibility effects are caused by automatic response activation by stimulus properties or by interference between codes during translation of stimulus into response coordinates. The main evidence against activation has been that in a Simon task with hands crossed, responses are faster at the response location ipsilateral to the stimulus though manipulated by the hand contralateral to the stimulus. The experiments were conducted with hands in standard and in crossed positions and electroencephalogram measures showed coactivation of the motor cortex induced by stimulus position primarily during standard hand positions with visual stimuli. Only in this condition did the Simon effect decay with longer response times. The visual Simon effect appeared to be due to specific mechanisms of visuomotor information transmission that are not responsible for the effects obtained with crossed hands or auditory stimuli.     

Transfer effects of incompatible location-relevant mappings on a subsequent visual or auditory Simon task

Two experiments investigated the influence of practice with an incompatible mapping of left and right stimuli to keypress responses on performance of a subsequent Simon task, for which stimulus location was irrelevant, after a delay of 5 min or 1 week. In Experiment 1, the visual Simon effect was eliminated when the practice modality was auditory and reversed to favor noncorresponding responses when it was visual, and there was no significant effect of delay interval. In Experiment 2, significant auditory Simon effects were obtained that did not vary as a function of practice modality, with delay having only a marginal effect on the magnitude of the Simon effect. The elimination of the visual Simon effect in the transfer session is most likely due to the short-term stimulus-response associations defined for the incompatible spatial mapping remaining active during the transfer session. Because the auditory Simon effect is stronger than the visual one, more practice with the incompatible mapping may be necessary to produce reliable transfer effects for it.     

Time course of the blindness to response-compatible stimuli

This article examines the time course of a deficit in identifying a stimulus sharing a compatible feature with a response that is executed in parallel ("blindness to response-compatible stimuli," J. Müsseler & B. Hommel, 1997a). In 5 experiments, participants performed a timed response, and the presentation point of time of a to-be-identified stimulus was varied in respect to response execution. A blindness effect was observed when the stimulus was presented between response cue offset and response execution. In contrast, the identification of a stimulus presented before the response cue or after response execution was not affected by stimulus-response compatibility--a finding that rules out a retention-based explanation. These results support an explanation that states that the perceptual processing of a stimulus feature is impaired as long as the shared perception-action feature code is integrated into the representation of a to-be-executed response.     

Between-task transfer of learning from spatial compatibility to a color stroop task

Responses to a relevant stimulus dimension are faster and more accurate when the stimulus and response spatially correspond compared to when they do not, even though stimulus position is irrelevant (Simon effect). It has been demonstrated that practicing with an incompatible spatial stimulus-response (S-R) mapping before performing a Simon task can eliminate this effect. In the present study we assessed whether a learned spatially incompatible S-R mapping can be transferred to a nonspatial conflict task, hence supporting the view that transfer effects are due to acquisition of a general "respond to the opposite stimulus value" rule. To this aim, we ran two experiments in which participants performed a spatial compatibility task with either a compatible or an incompatible mapping and then transferred, after a 5 min delay, to a color Stroop task. In Experiment 1, responses were executed by pressing one of two keys on the keyboard in both practice and transfer tasks. In Experiment 2, responses were manual in the practice task and vocal in the transfer task. The spatially incompatible practice significantly reduced the color Stroop effect only when responses were manual in both tasks. These results suggest that during practice participants develop a response-selection strategy of emitting the alternative spatial response.     

Age differences in response selection for pure and mixed stimulus-response mappings and tasks

Two experiments examined effects of mixed stimulus-response mappings and tasks for older and younger adults. In Experiment 1, participants performed two-choice spatial reaction tasks with blocks of pure and mixed compatible and incompatible mappings. In Experiment 2, a compatible or incompatible mapping was mixed with a Simon task for which the mapping of stimulus color to location was relevant and stimulus location was irrelevant. In both experiments, older adults showed larger mixing costs than younger adults and larger compatibility effects, with the differences particularly pronounced in Experiment 1 when location mappings were mixed. In mixed conditions, when stimulus location was relevant, older adults benefited more than younger adults from complete repetition of the task and stimulus from the preceding trial. When stimulus location was irrelevant, the benefit of complete repetition did not differ reliably between age groups. The results suggest that the age-related deficit associated with mixing mappings and tasks is primarily due to older adults having more difficulty separating task sets that activate conflicting response codes.