When speech enhances Spatial Musical Association of Response Codes: Joint spatial associations of pitch and timbre in nonmusicians

Previous studies have shown that the effect of the Spatial Musical Association of Response Codes (SMARC) depends on various features, such as task conditions (whether pitch height is implicit or explicit), response dimension (horizontal vs. vertical), presence or absence of a reference tone, and former musical training of the participants. In the present study, we investigated the effects of pitch range and timbre: in particular, how timbre (piano vs. vocal) contributes to the horizontal and vertical SMARC effect in nonmusicians under varied pitch range conditions. Nonmusicians performed a timbre judgement task in which the pitch range was either small (6 or 8 semitone steps) or large (9 or 12 semitone steps) in a horizontal and a vertical response setting. For piano sounds, SMARC effects were observed in all conditions. For the vocal sounds, in contrast, SMARC effects depended on pitch range. We concluded that the occurrence of the SMARC effect, especially in horizontal response settings, depends on the interaction of the timbre (vocal and piano) and pitch range if vocal and instrumental sounds are combined in one experiment: the human voice enhances the attention, both to the vocal and the instrumental sounds.     

Influence of auditory and audiovisual stimuli on the right–left prevalence effect

When auditory stimuli are used in two-dimensional spatial compatibility tasks, where the stimulus and response configurations vary along the horizontal and vertical dimensions simultaneously, a right–left prevalence effect occurs in which horizontal compatibility dominates over vertical compatibility. The right–left prevalence effects obtained with auditory stimuli are typically larger than that obtained with visual stimuli even though less attention should be demanded from the horizontal dimension in auditory processing. In the present study, we examined whether auditory or visual dominance occurs when the two-dimensional stimuli are audiovisual, as well as whether there will be cross-modal facilitation of response selection for the horizontal and vertical dimensions. We also examined whether there is an additional benefit of adding a pitch dimension to the auditory stimulus to facilitate vertical coding through use of the spatial-musical association of response codes (SMARC) effect, where pitch is coded in terms of height in space. In Experiment 1, we found a larger right–left prevalence effect for unimodal auditory than visual stimuli. Neutral, non-pitch coded, audiovisual stimuli did not result in cross-modal facilitation, but did show evidence of visual dominance. The right–left prevalence effect was eliminated in the presence of SMARC audiovisual stimuli, but the effect influenced horizontal rather than vertical coding. Experiment 2 showed that the influence of the pitch dimension was not in terms of influencing response selection on a trial-to-trial basis, but in terms of altering the salience of the task environment. Taken together, these findings indicate that in the absence of salient vertical cues, auditory and audiovisual stimuli tend to be coded along the horizontal dimension and vision tends to dominate audition in this two-dimensional spatial stimulus–response task.     

Effects of laterality and pitch height of an auditory accessory stimulus on horizontal response selection: The Simon effect and the SMARC effect

In the present article, we investigated the effects of pitch height and the presented ear (laterality) of an auditory stimulus, irrelevant to the ongoing visual task, on horizontal response selection. Performance was better when the response and the stimulated ear spatially corresponded (Simon effect), and when the spatial—musical association of response codes (SMARC) correspondence was maintained—that is, right (left) response with a high-pitched (low-pitched) tone. These findings reveal an automatic activation of spatially and musically associated responses by task-irrelevant auditory accessory stimuli. Pitch height is strong enough to influence the horizontal responses despite modality differences with task target.     

Transfer of orthogonal stimulus-response mappings to an orthogonal Simon task

When up-down stimulus locations are mapped to left-right keypresses, an overall advantage for the up-right/down-left mapping is often obtained that varies as a function of response eccentricity. This orthogonal stimulus-response compatibility (SRC) effect also occurs when stimulus location is irrelevant, a phenomenon called the orthogonal Simon effect, and has been attributed to correspondence of stimulus and response code polarities. The Simon effect for horizontal stimulus-response (S-R) arrangements has been shown to be affected by short-term S-R associations established through the mapping used for a prior SRC task in which stimulus location was relevant. We examined whether such associations also transfer between orthogonal SRC and Simon tasks and whether correspondence of code polarities continues to contribute to performance in the Simon task. In Experiment 1, the orthogonal Simon effect was larger after practising with an up-right/down-left mapping of visual stimuli to responses than with the alternative mapping, for which the orthogonal Simon effect tended to reverse. Experiment 2 showed similar results when practice was with high (up) and low (down) pitch tones, though the influence of practice mapping was not as large as that in Experiment 1, implying that the short-term S-R associations acquired in practice are at least in part not modality specific. In Experiment 3, response eccentricity and practice mapping were shown to have separate influences on the orthogonal Simon effect, as expected if both code polarity and acquired S-R associations contribute to performance.     

Transfer of orthogonal stimulus–response mappings to an orthogonal Simon task

When up–down stimulus locations are mapped to left–right keypresses, an overall advantage for the up–right/down–left mapping is often obtained that varies as a function of response eccentricity. This orthogonal stimulus–response compatibility (SRC) effect also occurs when stimulus location is irrelevant, a phenomenon called the orthogonal Simon effect, and has been attributed to correspondence of stimulus and response code polarities. The Simon effect for horizontal stimulus–response (S–R) arrangements has been shown to be affected by short-term S–R associations established through the mapping used for a prior SRC task in which stimulus location was relevant. We examined whether such associations also transfer between orthogonal SRC and Simon tasks and whether correspondence of code polarities continues to contribute to performance in the Simon task. In Experiment 1, the orthogonal Simon effect was larger after practising with an up–right/down–left mapping of visual stimuli to responses than with the alternative mapping, for which the orthogonal Simon effect tended to reverse. Experiment 2 showed similar results when practice was with high (up) and low (down) pitch tones, though the influence of practice mapping was not as large as that in Experiment 1, implying that the short-term S–R associations acquired in practice are at least in part not modality specific. In Experiment 3, response eccentricity and practice mapping were shown to have separate influences on the orthogonal Simon effect, as expected if both code polarity and acquired S–R associations contribute to performance.     

The Spatial Musical Association of Response Codes does not depend on a normal visual experience: A study with early blind individuals

Converging evidence suggests that the perception of auditory pitch exhibits a characteristic spatial organization. This pitch–space association can be demonstrated experimentally by the Spatial Musical Association of Response Codes (SMARC) effect. This is characterized by faster response times when a low-positioned key is pressed in response to a low-pitched tone, and a high-positioned key is pressed in response to a high-pitched tone. To investigate whether the development of this pitch–space association is mediated by normal visual experience, we tested a group of early blind individuals on a task that required them to discriminate the timbre of different instrument sounds with varying pitch. Results revealed a comparable pattern in the SMARC effect in both blind participants and sighted controls, suggesting that the lack of prior visual experience does not prevent the development of an association between pitch height and vertical space.     

Orthogonal stimulus–response compatibility effects emerge even when the stimulus position is task irrelevant

The above-right/below-left mapping advantage with vertical stimuli and horizontal responses is known as the orthogonal stimulus–response compatibility (SRC) effect. We investigated whether the orthogonal SRC effect emerges with irrelevant stimulus dimensions. In Experiment 1, participants responded with a right or left key press to the colour of the stimulus presented above or below the fixation. We observed an above-right/below-left advantage (orthogonal Simon effect). In Experiment 2, we manipulated the polarity in the response dimension by varying the horizontal location of the response set. The orthogonal Simon effect decreased and even reversed as the left response code became more positive. This result provides evidence for the automatic activation of the positive and negative response codes by the corresponding positive and negative stimulus codes. These findings extended the orthogonal SRC effect based on coding asymmetry to an irrelevant stimulus dimension.     

An investigation of spatial representation of pitch in individuals with congenital amusia

Spatial representation of pitch plays a central role in auditory processing. However, it is unknown whether impaired auditory processing is associated with impaired pitch–space mapping. Experiment 1 examined spatial representation of pitch in individuals with congenital amusia using a stimulus–response compatibility (SRC) task. For amusic and non-amusic participants, pitch classification was faster and more accurate when correct responses involved a physical action that was spatially congruent with the pitch height of the stimulus than when it was incongruent. However, this spatial representation of pitch was not as stable in amusic individuals, revealed by slower response times when compared with control individuals. One explanation is that the SRC effect in amusics reflects a linguistic association, requiring additional time to link pitch height and spatial location. To test this possibility, Experiment 2 employed a colour-classification task. Participants judged colour while ignoring a concurrent pitch by pressing one of two response keys positioned vertically to be congruent or incongruent with the pitch. The association between pitch and space was found in both groups, with comparable response times in the two groups, suggesting that amusic individuals are only slower to respond to tasks involving explicit judgments of pitch.     

Orthogonal stimulus-response compatibility effects emerge even when the stimulus position is task irrelevant

The above-right/below-left mapping advantage with vertical stimuli and horizontal responses is known as the orthogonal stimulus-response compatibility (SRC) effect. We investigated whether the orthogonal SRC effect emerges with irrelevant stimulus dimensions. In Experiment 1, participants responded with a right or left key press to the colour of the stimulus presented above or below the fixation. We observed an above-right/below-left advantage (orthogonal Simon effect). In Experiment 2, we manipulated the polarity in the response dimension by varying the horizontal location of the response set. The orthogonal Simon effect decreased and even reversed as the left response code became more positive. This result provides evidence for the automatic activation of the positive and negative response codes by the corresponding positive and negative stimulus codes. These findings extended the orthogonal SRC effect based on coding asymmetry to an irrelevant stimulus dimension.     

Representing response position relative to display location: Influence on orthogonal stimulus-response compatibility

Two types of stimulus-response compatibility (SRC) effect occur with orthogonal stimulus and response sets, an overall up-right/down-left advantage and mapping preferences that vary with response position. Researchers agree that the former type is due to asymmetric coding of the stimulus and response alternatives, but disagree as to whether the latter type requires a different explanation in terms of the properties of the motor system. This issue is examined in three experiments. The location of the stimulus set influenced orthogonal SRC when it varied along the same dimension as the responses (Experiments 1 and 2), with the pattern predicted by the hypothesis that the stimulus set provides a referent relative to which response position is coded. The effect of stimulus-set location on orthogonal SRC was independent of the stimulus onset asynchrony (SOA) for a marker that indicated stimulus-set side and the imperative stimulus. In contrast, a spatial correspondence effect for the irrelevant stimulus-set location and response was a decreasing function of SOA. Experiment 3 showed that the orthogonal SRC effect was determined by response position relative to the stimulus-set location and not the body midline. The results support the view that both types of orthogonal SRC effects are due to asymmetric coding of the stimuli and responses.     

Auditory stimulus-response compatibility: Is there a contribution of stimulus-hand correspondence?

 Simon, Hinrichs, and Craft found that when subjects responded to a tone in the left or right ear with a left or right keypress, both ear-response-location correspondence and ear-hand correspondence affected reaction time. This outcome is in contrast to results obtained for auditory and visual Simon tasks (i.e., tasks in which stimulus location is irrelevant) as well as results obtained in visual stimulus-response (S-R) compatibility studies, which show only an effect of spatial S-R correspondence. Experiment 1 was a replication of Simon et al.'s experiment in which spatial mapping and hand placement (uncrossed, crossed) were varied. The results were inconsistent with those of Simon et al., showing no ear-hand compatibility effect. Experiment 2 was a second replication with an additional condition examined in which the stimuli were visual locations. The results showed no contribution of stimulus-hand correspondence for either auditory or visual stimuli. Experiment 3 was a replication of another experiment by Simon et al. in which tone pitch was relevant and tone location irrelevant. Like Simon et al.'s data, our results showed no indication that stimulus-hand correspondence is a significant factor. Overall, our results imply that regardless of whether tone location is relevant or irrelevant, ear-response-location correspondence is the only factor that contributes to S-R compatibility in auditory two-choice reaction tasks. Received: 15 March 1999 / Accepted: 8 June 1999     

Display-control arrangement correspondence and logical recoding in the Hedge and Marsh reversal of the Simon effect

When left and right keypresses are made to stimuli in left and right locations, and stimulus location is irrelevant to the task, responses are typically faster when stimulus location corresponds with response location than when it does not (the Simon effect). This effect reverses when the relevant stimulus-response mapping is incompatible, with responses being slower when stimulus and response locations correspond (the Hedge and Marsh reversal). Simon et al. (Acta Psychol. 47 (1981) 63) reported an exception to the Hedge and Marsh reversal for a situation in which the relevant stimulus dimension was the color of a centered visual stimulus and the irrelevant location information was left or right tone location. In contrast, similar experiments have found a reversal of the Simon effect for tone location when relevant visual locations were mapped incompatibly to responses. We conducted four experiments to investigate this discrepancy. Both results were replicated. With an incompatible mapping, irrelevant tone location showed a small reverse Simon effect when the relevant visual dimension was physical location but not when the color of a centered stimulus or the direction in which an arrow pointed conveyed the visual location information. The reversal occurred in a more standard Hedge and Marsh task in which the irrelevant dimension was location of the colored stimulus, but only when the response keys were visibly labeled. Several of the results suggest that display-control arrangement correspondence is the primary cause of the Hedge and Marsh reversal, with logical recoding playing only a secondary role.     

Processing of irrelevant location information under dual-task conditions

This study deals with the problem of whether the processing of irrelevant location information in Simon-like tasks is triggered exogenously or endogenously. In Experiment 1, the primary task required one to press, as fast as possible, a left-hand-side key or a right-hand-side key (R1) to the pitch of a tone that was presented binaurally (S1). The secondary task required identifying, without time constraints, a visual stimulus (S2) that appeared randomly to the left or right of screen center. Results showed that there was a correspondence (i.e., a cross-task Simon effect) between the location of R1 and the location of S2 when S2 was presented alone. The cross-task Simon effect became much smaller (and in-significant) when a noise stimulus was presented contralateral to S2. Experiment 2 was similar to Experiment 1, except that S2 appeared unpredictably in only one-third of the trials. Results of Experiment 2 closely replicated those of Experiment 1: the cross-task Simon effect was much greater when S2 was presented alone. Experiment 3 differed from Experiment 1 because S2 had to be processed in only one-third of the trials, in which its identity was to be reported. In the remaining two-thirds of the trials, participants could ignore S2. Results confirmed that the cross-task Simon effect was much greater when S2 was presented alone. In contrast, it did not matter whether S2 had to be processed or not. In conclusion, the present study supports the hypothesis that the task-irrelevant spatial code of the stimulus is formed automatically, likely through an exogenously triggered selection. The role of endogenously initiated selection, if any, is much less important.     

The enhanced Simon effect for older adults is reduced when the irrelevant location information is conveyed by an accessory stimulus

The Simon effect, better performance when irrelevant stimulus location corresponds with the response location than when it does not, typically is larger for older than younger adults. However, Simon and Pouraghabagher [Simon, R. J., & Pouraghabagher, A. R. (1978). The effect of aging on the stages of processing in a choice reaction time task. Journal of Gerontology, 33, 553-561] found no age difference using an accessory-stimulus Simon task in which the relevant dimension was the color of a visual stimulus and the irrelevant dimension the location of a tone. Experiment 1 confirmed that older adults show a larger Simon effect than younger adults for the visual Simon task and that this age-related deficit is reduced or eliminated for the auditory-accessory task. Experiment 2 provided evidence suggesting that a small part of the age-related deficit in the visual Simon task is due to having to code the location of the relevant stimulus, but Experiment 3 showed that the majority of the deficit is due to the relevant and irrelevant information being conveyed by the same stimulus. Reaction-time distribution analyses show similar functions for younger and older adults, suggesting that the time course of activation is similar for both age groups.     

Tone‐affect compatibility with affective stimuli and affective responses

Three experiments examine multimodal integration of tone pitch (high/low), facial expression stimuli (happy/angry), and responses (happy/angry) in a compatibility paradigm. When the participants' task is to imitate facial expressions (Experiment 1), smiles are facilitated by high tones whereas frowns are facilitated by low tones. Experiments 2 and 3 further analyse this effect and show that there is both integration between the tone stimulus and the facial stimulus and between the tone stimulus and the facial response. Results suggest that pitch height is associated with emotion. An interpretation in terms of an embodied cognition approach that emphasizes an interweavement of perception and action is discussed.     

Between-task transfer of learning from spatial compatibility to a color stroop task

Responses to a relevant stimulus dimension are faster and more accurate when the stimulus and response spatially correspond compared to when they do not, even though stimulus position is irrelevant (Simon effect). It has been demonstrated that practicing with an incompatible spatial stimulus-response (S-R) mapping before performing a Simon task can eliminate this effect. In the present study we assessed whether a learned spatially incompatible S-R mapping can be transferred to a nonspatial conflict task, hence supporting the view that transfer effects are due to acquisition of a general "respond to the opposite stimulus value" rule. To this aim, we ran two experiments in which participants performed a spatial compatibility task with either a compatible or an incompatible mapping and then transferred, after a 5 min delay, to a color Stroop task. In Experiment 1, responses were executed by pressing one of two keys on the keyboard in both practice and transfer tasks. In Experiment 2, responses were manual in the practice task and vocal in the transfer task. The spatially incompatible practice significantly reduced the color Stroop effect only when responses were manual in both tasks. These results suggest that during practice participants develop a response-selection strategy of emitting the alternative spatial response.     

Influences of response position and hand posture on the orthogonal Simon effect

When lateralized responses are made to the locations of vertically arrayed stimuli, two types of mapping effect have been reported: an overall up-right/down-left advantage and mapping preferences that vary with response position. According to Cho and Proctor's (2003) multiple asymmetric codes account, these orthogonal stimulus-response compatibility effects are due to the correspondence of stimulus polarity and response polarity, as determined by the positions relative to multiple frames of reference. The present study examined these two types of orthogonal compatibility for situations in which participants made left-right responses to the colours of a vertically arrayed stimulus set, and stimulus location was irrelevant. Although a significant orthogonal Simon effect was not evident when responding at a centred, neutral response position, the effect was modulated by response eccentricity (Experiment 2) and hand posture (Experiment 3). These effects are qualitatively similar to those obtained when stimulus location is task relevant. The results imply that, as Proctor and Cho's (2006) polarity correspondence principle suggests, the stimulus polarity code activates the response code of corresponding polarity even when stimulus location is irrelevant to the task.     

Age differences in response selection for pure and mixed stimulus-response mappings and tasks

Two experiments examined effects of mixed stimulus-response mappings and tasks for older and younger adults. In Experiment 1, participants performed two-choice spatial reaction tasks with blocks of pure and mixed compatible and incompatible mappings. In Experiment 2, a compatible or incompatible mapping was mixed with a Simon task for which the mapping of stimulus color to location was relevant and stimulus location was irrelevant. In both experiments, older adults showed larger mixing costs than younger adults and larger compatibility effects, with the differences particularly pronounced in Experiment 1 when location mappings were mixed. In mixed conditions, when stimulus location was relevant, older adults benefited more than younger adults from complete repetition of the task and stimulus from the preceding trial. When stimulus location was irrelevant, the benefit of complete repetition did not differ reliably between age groups. The results suggest that the age-related deficit associated with mixing mappings and tasks is primarily due to older adults having more difficulty separating task sets that activate conflicting response codes.     

Does right-left prevalence occur for the Simon effect?

In four experiments, we investigated whether a right-left prevalence effect occurs for the Simon task, in which stimulus location is irrelevant, when the stimulus and the response sets vary along horizontal and vertical dimensions simultaneously. Simon effects were evident for both dimensions, and they were of similar magnitude, indicating no prevalence effect. Manipulations of the relative salience of the dimensions for the stimulus and the response sets resulted in a larger Simon effect for the more salient dimension than for the less salient one, but there was no overall prevalence effect. The results indicate that manipulations of salience affect the relative magnitudes of automatic response activation for the vertical and the horizontal dimensions but that the right-left prevalence effect is due to a coding bias in intentional response selection processes when stimulus location is relevant.     

Influences of response position and hand posture on the orthogonal Simon effect

When lateralized responses are made to the locations of vertically arrayed stimuli, two types of mapping effect have been reported: an overall up–right/down–left advantage and mapping preferences that vary with response position. According to Cho and Proctor's (2003) multiple asymmetric codes account, these orthogonal stimulus–response compatibility effects are due to the correspondence of stimulus polarity and response polarity, as determined by the positions relative to multiple frames of reference. The present study examined these two types of orthogonal compatibility for situations in which participants made left–right responses to the colours of a vertically arrayed stimulus set, and stimulus location was irrelevant. Although a significant orthogonal Simon effect was not evident when responding at a centred, neutral response position, the effect was modulated by response eccentricity (Experiment 2) and hand posture (Experiment 3). These effects are qualitatively similar to those obtained when stimulus location is task relevant. The results imply that, as Proctor and Cho's (2006) polarity correspondence principle suggests, the stimulus polarity code activates the response code of corresponding polarity even when stimulus location is irrelevant to the task.