The Spatial Musical Association of Response Codes does not depend on a normal visual experience: A study with early blind individuals

Converging evidence suggests that the perception of auditory pitch exhibits a characteristic spatial organization. This pitch–space association can be demonstrated experimentally by the Spatial Musical Association of Response Codes (SMARC) effect. This is characterized by faster response times when a low-positioned key is pressed in response to a low-pitched tone, and a high-positioned key is pressed in response to a high-pitched tone. To investigate whether the development of this pitch–space association is mediated by normal visual experience, we tested a group of early blind individuals on a task that required them to discriminate the timbre of different instrument sounds with varying pitch. Results revealed a comparable pattern in the SMARC effect in both blind participants and sighted controls, suggesting that the lack of prior visual experience does not prevent the development of an association between pitch height and vertical space.     

Referential coding contributes to the horizontal SMARC effect

The present study tested whether coding of tone pitch relative to a referent contributes to the correspondence effect between the pitch height of an auditory stimulus and the location of a lateralized response. When left-right responses are mapped to high or low pitch tones, performance is better with the high-right/low-left mapping than with the opposite mapping, a phenomenon called the horizontal SMARC effect. However, when pitch height is task irrelevant, the horizontal SMARC effect occurs only for musicians. In Experiment 1, nonmusicians performed a pitch discrimination task, and the SMARC effect was evident regardless of whether a referent tone was presented. However, in Experiment 2, for a timbre-judgment task, nonmusicians showed a SMARC effect only when a referent tone was presented, whereas musicians showed a SMARC effect that did not interact with presence/absence of the referent. Dependence of the SMARC effect for nonmusicians on a reference tone was replicated in Experiment 3, in which judgments of the color of a visual stimulus were made in the presence of a concurrent high- or low-pitched pure tone. These results suggest that referential coding of pitch height is a key determinant for the horizontal SMARC effect when pitch height is irrelevant to the task.     

Influence of auditory and audiovisual stimuli on the right–left prevalence effect

When auditory stimuli are used in two-dimensional spatial compatibility tasks, where the stimulus and response configurations vary along the horizontal and vertical dimensions simultaneously, a right–left prevalence effect occurs in which horizontal compatibility dominates over vertical compatibility. The right–left prevalence effects obtained with auditory stimuli are typically larger than that obtained with visual stimuli even though less attention should be demanded from the horizontal dimension in auditory processing. In the present study, we examined whether auditory or visual dominance occurs when the two-dimensional stimuli are audiovisual, as well as whether there will be cross-modal facilitation of response selection for the horizontal and vertical dimensions. We also examined whether there is an additional benefit of adding a pitch dimension to the auditory stimulus to facilitate vertical coding through use of the spatial-musical association of response codes (SMARC) effect, where pitch is coded in terms of height in space. In Experiment 1, we found a larger right–left prevalence effect for unimodal auditory than visual stimuli. Neutral, non-pitch coded, audiovisual stimuli did not result in cross-modal facilitation, but did show evidence of visual dominance. The right–left prevalence effect was eliminated in the presence of SMARC audiovisual stimuli, but the effect influenced horizontal rather than vertical coding. Experiment 2 showed that the influence of the pitch dimension was not in terms of influencing response selection on a trial-to-trial basis, but in terms of altering the salience of the task environment. Taken together, these findings indicate that in the absence of salient vertical cues, auditory and audiovisual stimuli tend to be coded along the horizontal dimension and vision tends to dominate audition in this two-dimensional spatial stimulus–response task.     

Binding “what” and “where” in auditory working memory: An asymmetrical association between sound identity and sound location

Contrasting results in visual and auditory working memory studies suggest that the mechanisms of association between location and identity of stimuli depend on the sensory modality of the input. In this auditory study, we tested whether the association of two features both encoded in the “what” stream is different from the association between a “what” and a “where” feature. In an old–new recognition task, blindfolded participants were presented with sequences of sounds varying in timbre, pitch and location. They were required to judge if either the timbre, pitch or location of a single-probe stimulus was identical or different to the timbre, pitch or location of one of the sounds of the previous sequence. Only variations in one of the three features were relevant for the task, whereas the other two features could vary, with task-irrelevant changes. Results showed that task-irrelevant variations in the “what” features (either timbre or pitch) caused an impaired recognition of sound location and in the other task-relevant “what” feature, whereas changes in sound location did not affect the recognition of either one of the “what” features. We conclude that the identity of sounds is incidentally processed even when not required by the task, whereas sound location is not maintained when task irrelevant.     

Effects of laterality and pitch height of an auditory accessory stimulus on horizontal response selection: The Simon effect and the SMARC effect

In the present article, we investigated the effects of pitch height and the presented ear (laterality) of an auditory stimulus, irrelevant to the ongoing visual task, on horizontal response selection. Performance was better when the response and the stimulated ear spatially corresponded (Simon effect), and when the spatial—musical association of response codes (SMARC) correspondence was maintained—that is, right (left) response with a high-pitched (low-pitched) tone. These findings reveal an automatic activation of spatially and musically associated responses by task-irrelevant auditory accessory stimuli. Pitch height is strong enough to influence the horizontal responses despite modality differences with task target.     

声乐与器乐情绪加工的ERP研究

借助跨通道情绪启动范式考察声乐与器乐情绪加工的电生理差异。启动刺激是声乐和器乐曲(小提琴演奏), 目标刺激是与音乐情绪一致或不一致的面孔表情图片。结果显示, 与一致条件相比, 与面孔情绪不一致的器乐曲诱发了N400, 而与面孔情绪不一致的声乐曲诱发了LPC。这些结果表明, 声乐与器乐情绪加工的电生理反应存在差异。     

The relative influence of pitch and timbre on the apparent location of sound in the median sagittal plane

Listeners were requested to locate sounds originating in the median sagittal plane (MSP). The stimuli, 75-Msec pulses, were repeated at a rate of 200 times/sec and were filtered to transmit narrow bands centered about 0.63, 1.6, 2.5, or 6.3 kHz. Despite the sameness of pitch generated by all stimuli, Os perceived the 0.63-, 1.6-, and 2.5-kHz-centered sounds as originating low, middle, and high, respectively, in the MSP, regardless of their actual positions. The stimuli centered about 6.3 kHz, on the other hand, were located accurately by most Os. These findings were interpreted to mean that under conditions of inadequate auditory cues, timbre, not pitch, influences perceived elevation. The implication is that timbre also served as the cue for the apparent elevation of those sounds which, due to their high-frequency components, can be located accurately.     

Effects of differences in timbre on sequential grouping

Differences in the timbre of sounds in a sequence can affect their perceptual organization. Using a performance measure, Hartmann and Johnson (1991) concluded that streaming could be predicted primarily by the extent to which sounds were passed by different peripheral channels. However, results from a rating task by Dannenbring and Bregman (1976) suggested that sounds in the same spectral region (passed by the same peripheral channels) can be allocated to different streams. In Experiment 1, it was found, using an interleaved melody task, that target sounds could be selected from distractors in the same spectral region more easily when they differed in timbre. This finding might result from primitive stream segregation or schema-driven selection, but not from peripheral channeling. In Experiment 2, a rhythm discrimination task was used, requiring the sounds to be integrated for good performance. Differences in timbre impaired performance, indicating the occurrence of primitive stream segregation.     

When infants talk,infants listen: pre‐babbling infants prefer listening to speech with infant vocal properties

To learn to produce speech, infants must effectively monitor and assess their own speech output. Yet very little is known about how infants perceive speech produced by an infant, which has higher voice pitch and formant frequencies compared to adult or child speech. Here, we tested whether pre‐babbling infants (at 4–6 months) prefer listening to vowel sounds with infant vocal properties over vowel sounds with adult vocal properties. A listening preference favoring infant vowels may derive from their higher voice pitch, which has been shown to attract infant attention in infant‐directed speech (IDS). In addition, infants' nascent articulatory abilities may induce a bias favoring infant speech given that 4‐ to 6‐month‐olds are beginning to produce vowel sounds. We created infant and adult /i/ (‘ee’) vowels using a production‐based synthesizer that simulates the act of speaking in talkers at different ages and then tested infants across four experiments using a sequential preferential listening task. The findings provide the first evidence that infants preferentially attend to vowel sounds with infant voice pitch and/or formants over vowel sounds with no infant‐like vocal properties, supporting the view that infants' production abilities influence how they process infant speech. The findings with respect to voice pitch also reveal parallels between IDS and infant speech, raising new questions about the role of this speech register in infant development. Research exploring the underpinnings and impact of this perceptual bias can expand our understanding of infant language development.     

Pianists exhibit enhanced memory for vocal melodies but not piano melodies

Nonmusicians remember vocal melodies (i.e., sung to la la) better than instrumental melodies. If greater exposure to the voice contributes to those effects, then long-term experience with instrumental timbres should elicit instrument-specific advantages. Here we evaluate this hypothesis by comparing pianists with other musicians and nonmusicians. We also evaluate the possibility that absolute pitch (AP), which involves exceptional memory for isolated pitches, influences melodic memory. Participants heard 24 melodies played in four timbres (voice, piano, banjo, marimba) and were subsequently required to distinguish the melodies heard previously from 24 novel melodies presented in the same timbres. Musicians performed better than nonmusicians, but both groups showed a comparable memory advantage for vocal melodies. Moreover, pianists performed no better on melodies played on piano than on other instruments, and AP musicians performed no differently than non-AP musicians. The findings confirm the robust nature of the voice advantage and rule out explanations based on familiarity, practice, and motor representations.     

Effects of altered auditory feedback across effector systems: production of melodies by keyboard and singing

We report an experiment that tested whether effects of altered auditory feedback (AAF) during piano performance differ from its effects during singing. These effector systems differ with respect to the mapping between motor gestures and pitch content of auditory feedback. Whereas this action-effect mapping is highly reliable during phonation in any vocal motor task (singing or speaking), mapping between finger movements and pitch occurs only in limited situations, such as piano playing. Effects of AAF in both tasks replicated results previously found for keyboard performance (Pfordresher, 2003), in that asynchronous (delayed) feedback slowed timing whereas alterations to feedback pitch increased error rates, and the effect of asynchronous feedback was similar in magnitude across tasks. However, manipulations of feedback pitch had larger effects on singing than on keyboard production, suggesting effector-specific differences in sensitivity to action-effect mapping with respect to feedback content. These results support the view that disruption from AAF is based on abstract, effector independent, response-effect associations but that the strength of associations differs across effector systems.     

Harmonic,melodic, and frequency height influences in the perception of multivoiced music

Two experiments addressed the influences of harmonic relations, melody location, and relative frequency height on the perceptual organization of multivoiced music. In Experiment 1, listeners detected pitch changes in multivoiced piano music. Harmonically related pitch changes and those in the middle-frequency range were least noticeable. All pitch changes were noticeable in the high-frequency voice containing the melody (the most important voice), suggesting that melody can dominate harmonic relations. However, the presence of upper partials in the piano timbre used may have accounted for the harmonic effects. Experiment 2 employed pure sine tones, and replicated the effects of Experiment 1. In addition, the influence of the high-frequency melody on the noticeability of harmonically related pitches was lessened by the presence of a second melody. These findings suggest that harmonic, melodic, and relative frequency height relationships among voices interact in the perceptual organization of multivoiced music.     

Prospects for timbre physicalism

Timbre is that property of a sound that distinguishes it other than pitch and loudness, for instance the distinctive sound quality of a violin or flute. While the term is obscure, the concept has played an important, implicit role in recent philosophy of sound. Philosophers have debated whether to identify sounds with properties of waves, events, or objects. Many of the intuitive considerations in this debate apply most clearly to timbre qualities. Two prominent forms of timbre physicalism have emerged: one identifying timbre with the spectral composition of proximal waves; the second identifying timbre with the mechanical vibrations at a sound source. I demonstrate that the first possibility is conceptually unsatisfying, while the second fails to meet the standards of rigor established by the color physicalism literature. One response to these worries might be to adopt a more modest, non-reductive realism about timbre, such as the ecological view of J. J. Gibson.     

Implicit chord processing and motor representation in pianists

The aim of this paper is to assess the relevance of pitch dimension in auditory–motor interaction. Several behavioural and brain imaging studies have shown that auditory processing of sounds can activate motor representations, an effect which is however elicited only by action-related sounds, i.e., sounds linked to a specific motor repertoire. Music provides an appropriate framework for further exploration of this issue. Three groups of participants (pianists, non-pianist musicians and non-musicians) were tested with a shape decision task where left-hand and right-hand responses were required; each visual stimulus was paired with an auditory task-irrelevant stimulus (high-pitched or low-pitched piano-timbre chords). Of the three groups, only pianists had longer reaction times for left-hand/high-pitched chords and right-hand/low-pitched chords associations. These findings are consistent with an auditory-motor effect elicited by pitch dimension, as only pianists show an interaction between motor responses and implicit pitch processing. This interaction is consistent with the canonical mapping of hand gestures and pitch dimension on the piano keyboard. The results are discussed within the ideo-motor theoretical framework offered by the Theory of Event Coding (Hommel et al. in Behav Brain Sci 24:849–937, 2001).     

The influence of linguistic experience on the cognitive processing of pitch in speech and nonspeech sounds

In the present experiment, the authors tested Mandarin and English listeners on a range of auditory tasks to investigate whether long-term linguistic experience influences the cognitive processing of nonspeech sounds. As expected, Mandarin listeners identified Mandarin tones significantly more accurately than English listeners; however, performance did not differ across the listener groups on a pitch discrimination task requiring fine-grained discrimination of simple nonspeech sounds. The crucial finding was that cross-language differences emerged on a nonspeech pitch contour identification task: The Mandarin listeners more often misidentified flat and falling pitch contours than the English listeners in a manner that could be related to specific features of the sound structure of Mandarin, which suggests that the effect of linguistic experience extends to nonspeech processing under certain stimulus and task conditions.     

Psychoacoustic evaluation of musical sounds

A review is given of several basic aspects of musical sounds, i.e., the perception of duration, the perception of sound impulses as events within temporal patterns, timbre, the equivalence of sensational intervals, roughness, and the pitch qualities of complex sounds. Selected examples illustrate how psychoacoustic results can contribute to the evaluation of musical sounds and to the understanding of the perception of music.     

On the relation between roll and pitch

Studies have found that rolling the visual environment affects observers' perception of gravitational vertical and horizontal and that pitching the environment affects observers' perception of pitch. However, the relationship between these two perceptions is not fully understood. In the present work, observers performed three tasks while in a visual surround whose pitch and roll was manipulated. In the first task, observers adjusted a rod in the frontal-parallel plane to the horizontal (roll). In a second task, they adjusted a rod along a plane parallel to straight-ahead to the vertical (pitch). In the final task (“in-between”), they adjusted a rod midway between the first two conditions. The typical pitch and roll effects were found, as well as a contribution of both pitch and roll to the in-between task. No interaction between pitch and roll effects was found, indicating independent cognitive representations.     

Spatial associations for musical stimuli: a piano in the head?

This study was aimed at examining whether pitch height and pitch change are mentally represented along spatial axes. A series of experiments explored, for isolated tones and 2-note intervals, the occurrence of effects analogous to the spatial numerical association of response codes (SNARC) effect. Response device orientation (horizontal vs. vertical), task, and musical expertise of the participants were manipulated. The pitch of isolated tones triggered the automatic activation of a vertical axis independently of musical expertise, but the contour of melodic intervals did not. By contrast, automatic associations with the horizontal axis seemed linked to music training for pitch and, to a lower extent, for intervals. These results, discussed in the light of studies on number representation, provide a new example of the effects of musical expertise on music cognition.     

Cross-modal and intra-modal binding between identity and location in spatial working memory: The identity of objects does not help recalling their locations

In this study we tested incidental feature-to-location binding in a spatial task, both in unimodal and cross-modal conditions. In Experiment 1 we administered a computerised version of the Corsi Block-Tapping Task (CBTT) in three different conditions: the first one analogous to the original CBTT test; the second one in which locations were associated with unfamiliar images; the third one in which locations were associated with non-verbal sounds. Results showed no effect on performance by the addition of identity information. In Experiment 2, locations on the screen were associated with pitched sounds in two different conditions: one in which different pitches were randomly associated with locations and the other in which pitches were assigned to match the vertical position of the CBTT squares congruently with their frequencies. In Experiment 2 we found marginal evidence of a pitch facilitation effect in the spatial memory task. We ran a third experiment to test the same conditions of Experiment 2 with a within-subject design. Results of Experiment 3 did not confirm the pitch–location facilitation effect. We concluded that the identity of objects does not affect recalling their locations. We discuss our results within the framework of the debate about the mechanisms of “what” and “where” feature binding in working memory.     

Automatic response activation of implicit spatial information: Evidence from the SNARC effect

In the present paper, we focus on how irrelevant implicit spatial information is processed. By irrelevant we mean information that is not required to fulfill the task and by implicit we mean information that is not directly available in the external stimulus. A good example of a task in which such information exists is the SNARC task [Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122, 371-396]. The SNARC effect shows that the magnitude of a number, although irrelevant to the task, activates spatial codes that may interfere with the task-related response. These spatial associations exist both for the horizontal and the vertical direction. In Experiment 1, response keys were discriminating in the vertical or the horizontal direction. It is shown that the impact of the numerical spatial codes on overt behavior, although automatic, depends on the response discrimination of the horizontal or the vertical dimension. In Experiment 2, response keys were assigned such that both the horizontal and the vertical direction of the response were discriminating. In this case, the horizontal and the vertical dimension of the irrelevant numerical spatial codes were shown to interact. In general, the results are in line with the response-discrimination account [Ansorge, U., & Wühr, P. (2004). A response-discrimination account of the Simon effect. Journal of Experimental Psychology: Human Perception and Performance, 30, 365-377].