The spatial attributes of stimulus frequency and their role in monaural localization of sound in the horizontal plane

Listeners, whose right ears were blocked, located low-intensity sounds originating from loudspeakers placed 15 deg apart along the horizontal plane on the side of the open, or functioning, ear. In Experiment 1, the stimuli consisted of noise bursts, 1.0 kHz wide and centered at 4.0 through 14.0 kHz in steps of .5 kHz. We found that the apparent location of the noise bursts was governed by their frequency composition. Specifically, as the center frequency was increased from 4.0 to about 8.0 kHz, the sound appeared to move away from the frontal sector and toward the side. This migration pattern of the apparent sound source was observed again when the center frequency was increased from 8.0 to about 12.0 kHz. Then, with center frequencies of 13.0 and 14.0 kHz, the sound appeared once more in front. We referred to this relation between frequency composition and apparent location in terms of spatial referent maps. In Experiment 2, we showed that localization was more proficient if the frequency content of the stimulus served to connect adjacent spatial referent maps rather than falling within a single map. By these means, we have further elucidated the spectral cues utilized in monaural localization of sound in the horizontal plane.     

Binding “what” and “where” in auditory working memory: An asymmetrical association between sound identity and sound location

Contrasting results in visual and auditory working memory studies suggest that the mechanisms of association between location and identity of stimuli depend on the sensory modality of the input. In this auditory study, we tested whether the association of two features both encoded in the “what” stream is different from the association between a “what” and a “where” feature. In an old–new recognition task, blindfolded participants were presented with sequences of sounds varying in timbre, pitch and location. They were required to judge if either the timbre, pitch or location of a single-probe stimulus was identical or different to the timbre, pitch or location of one of the sounds of the previous sequence. Only variations in one of the three features were relevant for the task, whereas the other two features could vary, with task-irrelevant changes. Results showed that task-irrelevant variations in the “what” features (either timbre or pitch) caused an impaired recognition of sound location and in the other task-relevant “what” feature, whereas changes in sound location did not affect the recognition of either one of the “what” features. We conclude that the identity of sounds is incidentally processed even when not required by the task, whereas sound location is not maintained when task irrelevant.     

Spectral cues which influence monaural Localization in the horizontal plane

An extensive series of behavioral tests was carried out to determine what region, or regions, of the sound spectrum were critical for locating sounds monaurally in the horizontal plane. Seven subjects were requested to locate narrow bands of noise centered at different frequencies, combinations of these noise bands, low-pass, high-pass, and broadband noise. As observed in an earlier study, increasing bandwidth did not necessarily lead to improved localization performance until the band became broad, including, for example, all frequencies above 4.0 kHz. What seems to be happening is that listeners perceive narrow bands of noise originating from restricted places in the horizontal plane which may differ one from another depending on the frequency composition of the stimulus. In several instances, if two noise bands were presented simultaneously, the resulting stimulus was located with reasonable accuracy provided each component, when presented singly, was perceived as emanating from clearly separate azimuthal positions. If, however, two noise bands, which were perceived to originate from approximately the same azimuthal position when presented singly, were now presented simultaneously, the resulting stimulus still was perceived to originate from the same region of the horizontal plane. This, then, is a case where augmenting the spectral content of the stimulus does not bring about improved performance. We suggest that the expression of judgmental biases in the apparent location of a band of noise may prove useful for understanding why some stimuli of specified width and center frequency are localizable while others are not.     

An interval size illusion: the influence of timbre on the perceived size of melodic intervals

In four experiments, we investigated the influence of timbre on perceived interval size. In Experiment 1, musically untrained participants heard two successive tones and rated the pitch distance between them. Tones were separated by six or seven semitones and varied in timbre. Pitch changes were accompanied by a congruent timbre change (e.g., ascending interval involving a shift from a dull to a bright timbre), an incongruent timbre change (e.g., ascending interval involving a shift from a bright to a dull timbre), or no timbre change. Ratings of interval size were strongly influenced by timbre. The six-semitone interval with a congruent timbre change was perceived to be larger than the seven-semitone interval with an incongruent timbre change (interval illusion). Experiment 2 revealed similar effects for musically trained participants. In Experiment 3, participants compared the size of two intervals presented one after the other. Effects of timbre were again observed, including evidence of an interval illusion. Experiment 4 confirmed that timbre manipulations did not distort the perceived pitch of tones. Changes in timbre can expand or contract the perceived size of intervals without distorting individual pitches. We discuss processes underlying interval size perception and their relation to pitch perception mechanisms.     

When speech enhances Spatial Musical Association of Response Codes: Joint spatial associations of pitch and timbre in nonmusicians

Previous studies have shown that the effect of the Spatial Musical Association of Response Codes (SMARC) depends on various features, such as task conditions (whether pitch height is implicit or explicit), response dimension (horizontal vs. vertical), presence or absence of a reference tone, and former musical training of the participants. In the present study, we investigated the effects of pitch range and timbre: in particular, how timbre (piano vs. vocal) contributes to the horizontal and vertical SMARC effect in nonmusicians under varied pitch range conditions. Nonmusicians performed a timbre judgement task in which the pitch range was either small (6 or 8 semitone steps) or large (9 or 12 semitone steps) in a horizontal and a vertical response setting. For piano sounds, SMARC effects were observed in all conditions. For the vocal sounds, in contrast, SMARC effects depended on pitch range. We concluded that the occurrence of the SMARC effect, especially in horizontal response settings, depends on the interaction of the timbre (vocal and piano) and pitch range if vocal and instrumental sounds are combined in one experiment: the human voice enhances the attention, both to the vocal and the instrumental sounds.     

Spectral cues provided by the pinna for monaural localization in the horizontal plane

Six subjects located, monaurally, 1.0-kHz-wide noise bursts whose source originated on the side of the functioning ear and whose center frequency ranged from 4.0 through 9.0 kHz (Part 1). Irrespective of their actual locations, the stimuli appeared to migrate from the frontal sector of the arc toward the side as the center frequency was increased above 4.0 kHz. For some subjects, the sounds appeared again in front at the higher center frequencies. Comparable data were obtained with noise bursts 2.0 kHz in width. We referred to these constellations of location judgments, influenced by the frequency composition of the stimuli, as spatial referent maps. In Part 2, we measured, by means of a miniature microphone placed at the entrance of the external ear canal, the pinna amplification function for these same stimuli emanating from the same locations. The results showed a positive relation between the apparent location of noise bursts centered at 6.0 kHz and above and the relative amplification provided by the pinna. Localization performances by two subjects, chosen on the basis of their noncorresponding spatial referent maps, were examined for stimuli of wider bandwidths IPart 3). Their proficiency differed markedly from one another, which we accounted for in terms of different spatial referent maps that were associated with differences in the pinna amplification function.     

Perceptual interactions between musical pitch and timbre.

These experiments examined perceptual interactions between musical pitch and timbre. Experiment 1, through the use of the Garner classification tasks, found that pitch and timbre of isolated tones interact. Classification times showed interference from uncorrelated variation in the irrelevant attribute and facilitation from correlated variation; the effects were symmetrical. Experiments 2 and 3 examined how musical pitch and timbre function in longer sequences. In recognition memory tasks, a target tone always appeared in a fixed position in the sequences, and listeners were instructed to attend to either its pitch or its timbre. For successive tones, no interactions between timbre and pitch were found. That is, changing the pitches of context tones did not affect timbre recognition, and vice versa. The tendency to perceive pitch in relation to other context pitches was strong and unaffected by whether timbre was constant or varying. In contrast, the relative perception of timbre was weak and was found only when pitch was constant. These results suggest that timbre is perceived more in absolute than in relative terms. Perceptual implications for creating patterns in music with timbre variations are discussed.     

Psychoacoustic evaluation of musical sounds

A review is given of several basic aspects of musical sounds, i.e., the perception of duration, the perception of sound impulses as events within temporal patterns, timbre, the equivalence of sensational intervals, roughness, and the pitch qualities of complex sounds. Selected examples illustrate how psychoacoustic results can contribute to the evaluation of musical sounds and to the understanding of the perception of music.     

Asymmetrical Perception of Changing Intensity in Short Tonal Stimuli: Duration of Stimulus

A number of reports have suggested that changing intensity in short tonal stimuli is asymmetrically perceived. In particular, steady stimuli may be heard as growing louder; stimuli must decrease in intensity to be heard as steady in loudness. The influence of stimulus duration on this perceptual asymmetry was examined. Three participants heard diotic tonal stimuli of eight durations between 0.8 s and 2.5 s. Each stimulus increased, decreased, or remained steady in intensity; initial intensity was 40 dB SPL (sound pressure level relative to 0.0002 dynes/cm²), and carrier frequency was 1 kHz. Participants made forced binary responses of “growing louder” or “growing softer” to each stimulus. For each duration, that value of intensity change eliciting equal numbers of both responses was determined. The results indicated a pronounced perceptual asymmetry for 0.8-s stimuli, which diminished for longer stimuli; changing intensity in 2.5-s stimuli was perceived symmetrically. Additionally, sensitivity to changing intensity improved as stimulus duration increased, suggesting that responses may be based in part on the difference in intensity between the beginning and end of the stimulus. Possible ramifications of the asymmetry reside in (a) the percussive nature of many natural sounds and (b) selective responding to approaching sound sources.     

Prospects for timbre physicalism

Timbre is that property of a sound that distinguishes it other than pitch and loudness, for instance the distinctive sound quality of a violin or flute. While the term is obscure, the concept has played an important, implicit role in recent philosophy of sound. Philosophers have debated whether to identify sounds with properties of waves, events, or objects. Many of the intuitive considerations in this debate apply most clearly to timbre qualities. Two prominent forms of timbre physicalism have emerged: one identifying timbre with the spectral composition of proximal waves; the second identifying timbre with the mechanical vibrations at a sound source. I demonstrate that the first possibility is conceptually unsatisfying, while the second fails to meet the standards of rigor established by the color physicalism literature. One response to these worries might be to adopt a more modest, non-reductive realism about timbre, such as the ecological view of J. J. Gibson.     

Visually perceived eye level with reversible pitch stimuli: implications for the great circle and implicit surface models

Visually perceived eye level (VPEL) and perceived pitch were measured while subjects viewed two sets of stimuli that were either upright or pitched top-toward or top-away from them. The first set of stimuli, a pair of vertical lines viewed at various angles of pitch, caused systematic changes in perceived pitch and upward and downward VPEL shifts for the top-toward and top-away pitches, respectively. Neither the perceived pitch nor the VPEL measures with these stimuli differed between monocular and binocular viewing. The second set of stimuli was constructed so that, when viewed at the appropriate pitch angle, the projected orientations of the lines in the retinal image of each stimulus were similar to those generated by a pair of vertical lines pitched by a lesser amount in the opposite direction. When viewed monocularly, these stimuli appeared pitched in the direction opposite their physical pitch, yet produced VPEL shifts consistent with the direction of their physical pitch. These results clearly demonstrate a dissociation between perceived pitch and VPEL. The same stimuli, when viewed binocularly, appeared pitched in the direction of their physical pitch and caused VPEL shifts indistinguishable from those obtained monocularly. The retinal image orientations of these stimuli, however, corresponded to those of vertical line stimuli pitched in the opposite direction. This finding is therefore consistent with the hypothesis that VPEL and perceived pitch are processed independently, but inconsistent with the specific version of this hypothesis which states that differences in VPEL are determined solely on the basis of the orientation of lines in the retinal image.     

Change in visually perceived eye level without change in perceived pitch

Both the physical elevation that appears to correspond to eye level and the visually perceived pitch of a visual field are linear functions of the physical pitch of a normally illuminated, complexly structured visual field. One of the possible bases for the large effect of physical pitch on the elevation of visually perceived eye level (VPEL) is that the visual field generates a mental representation which specifies spatial coordinates and these determine the VPEL elevation ('implicit-surface model'; ISM). The influence on the elevation of VPEL is nearly as large when the visual field contains either one or two long pitched-from-vertical or rolled-from-vertical lines in otherwise total darkness as when it consists of a well-illuminated and complexly structured pitched room (L Matin and W Li, 1994 Vision Research 34 311-330), and, in order to examine the ISM, we employed a rolled-from-vertical, two-line configuration within a frontoparallel plane viewed in otherwise total darkness. Measurements of visually perceived pitch were made by a manual matching procedure and VPEL measurements were made by the psychophysical setting of the elevation of a small visual target to appear at eye level while each of three subjects viewed the two-line configuration at each of three horizontal eccentricities with the configuration at each of seven roll orientations. In direct contradiction to the ISM, the perceived pitch of the two-line configuration did not deviate significantly from the erect orientation ('vertical') for any roll at any eccentricity, but the elevation of VPEL changed systematically with the roll of the configuration both at left and at right eccentricities, and did not change at all with the two-line configuration centered on the median plane. Consistent with our previous work and with our previous interpretation regarding the basis for VPEL (L Matin and W Li, 1994 Vision Research 34 2577-2598), the variation of VPEL for the two-line visual field equals the average of the VPEL variations produced by viewing each of the single lines separately.     

The influence of stimulus bandwidth on localization of sound in space

The ability of listeners, deprived of prominent interaural time and intensity cues, to locate noise bands differing in width was investigated. To minimize binaural cues, we placed the sound source at various positions in the median sagittal plane. To eliminate binaural cues, we occluded one ear. The stimuli consisted of broadband noise and bands of noise centered at 8.0 kHz. The width of the latter ranged from 1.0 to 6.0 kHz. The results from seven listeners showed that localization proficiency for sounds in the median sagittal plane decreased with decreases in bandwidth for both binaural and monaural listening conditions. This function was less orderly for monaural localization of horizontally positioned sounds. Another consequence of a reduction in bandwidth was an increasing tendency of listeners to select certain loudspeakers over others as the source of the sound. A previous finding showing that localization of sound in the median sagittal plane is more accurate when listening binaurally rather than monaurally was confirmed.     

Evidence for a central representation of instrument timbre

An information processing account of perception seeks to delineate the stages of processing through which a stimulus passes and determine the properties of the representation at each stage. Research in phonetic perception has identified two stages, the second of which is thought to encode abstract acoustic attributes of sounds. The present study provided a further test of this proposal by assessing whether nonphonetic stimuli could yield results similar to those obtained with phonetic stimuli. Five selective adaptation experiments were carried out with a trumpet—piano timbre continuum. Two manipulations were used to measure abstract encoding: cross-ear presentation of adaptor and test series, and the use of adaptors that were acoustically different from the continuum endpoints. The results provide evidence for an abstract representation of timbre. The similarity of the findings to those in the phonetic adaptation literature is discussed.     

Flexible and abstract resolutions to crossmodal conflicts

Participants judged whether two sequential visual events were presented for the same length of time or for different lengths of time, while ignoring two irrelevant sequential sounds. Sounds could be either the same or different in terms of their duration or their pitch. When the visual stimuli were in conflict with the sound stimuli (e.g., visual events were the same, but the sounds were different) performance declined. This was true whether sounds varied in duration or in pitch. The influence of sounds was eliminated when visual duration discriminations were made easier. Together these results demonstrate that resolutions to crossmodal conflicts are flexible across the neural and cognitive architectures. More importantly, they suggest that interactions between modalities can span to abstract levels of same/different representations.     

The Spatial Musical Association of Response Codes does not depend on a normal visual experience: A study with early blind individuals

Converging evidence suggests that the perception of auditory pitch exhibits a characteristic spatial organization. This pitch–space association can be demonstrated experimentally by the Spatial Musical Association of Response Codes (SMARC) effect. This is characterized by faster response times when a low-positioned key is pressed in response to a low-pitched tone, and a high-positioned key is pressed in response to a high-pitched tone. To investigate whether the development of this pitch–space association is mediated by normal visual experience, we tested a group of early blind individuals on a task that required them to discriminate the timbre of different instrument sounds with varying pitch. Results revealed a comparable pattern in the SMARC effect in both blind participants and sighted controls, suggesting that the lack of prior visual experience does not prevent the development of an association between pitch height and vertical space.     

Constraints on the processes responsible for the extrinsic normalization of vowels

Listeners tune in to talkers’ vowels through extrinsic normalization. We asked here whether this process could be based on compensation for the long-term average spectrum (LTAS) of preceding sounds and whether the mechanisms responsible for normalization are indifferent to the nature of those sounds. If so, normalization should apply to nonspeech stimuli. Previous findings were replicated with first-formant (F1) manipulations of speech. Targets on a [pt]–[p?t] (low–high F1) continuum were labeled as [pt] more after high-F1 than after low-F1 precursors. Spectrally rotated nonspeech versions of these materials produced similar normalization. None occurred, however, with nonspeech stimuli that were less speechlike, even though precursor–target LTAS relations were equivalent to those used earlier. Additional experiments investigated the roles of pitch movement, amplitude variation, formant location, and the stimuli's perceived similarity to speech. It appears that normalization is not restricted to speech but that the nature of the preceding sounds does matter. Extrinsic normalization of vowels is due, at least in part, to an auditory process that may require familiarity with the spectrotemporal characteristics of speech.     

Brief pitch-priming facilitates infants’ discrimination of pitch-evoking noise: Evidence from event-related potentials

Pitch is derived by the auditory system through complex spectrotemporal processing. Pitch extraction is thought to depend on both spectral cues arising from lower harmonics that are resolved by cochlear filters in the inner ear, and on temporal cues arising from the pattern of action potentials contained in the cochlear output. Adults are capable of extracting pitch in the absence of robust spectral cues, taking advantage of the temporal cues that remain. However, recent behavioral evidence suggests that infants have difficulty discriminating between stimuli with different pitches when resolvable spectral cues are absent. In the current experiments, we used the mismatch negativity (MMN) component of the event related potential derived from electroencephalographic (EEG) recordings to examine a cortical representation of pitch discrimination for iterated rippled noise (IRN) stimuli in 4- and 8-month-old infants. IRN stimuli are pitch-evoking sounds generated by repeatedly adding a segment of white noise to itself at a constant delay. We created IRN stimuli (delays of 5 and 6 ms creating pitch percepts of 200 and 167 Hz) and high-pass filtered them to remove all resolvable spectral pitch cues. In experiment 1, we did not find EEG evidence that infants could detect the change in the pitch of these IRN stimuli. However, in Experiment 2, after a brief period of pitch-priming during which we added a sine wave component to the IRN stimulus at its perceived pitch, infants did show significant MMN in response to pitch changes in the IRN stimuli with sine waves removed. This suggests that (1) infants can use temporal cues to process pitch, although such processing is not mature and (2) that a short amount of pitch-priming experience can alter pitch representations in auditory cortex during infancy.     

Pitch perception in chinchillas (Chinchilla laniger): stimulus generalization using rippled noise

Rippled noises evoke the perception of pitch in human listeners. Infinitely iterated rippled noise (IIRN) is generated when wideband noise (WBN) is delayed, attenuated, and added to the original WBN through either a positive (+) or a negative (-) feedback loop. The pitch of IIRN[+] is matched to the reciprocal of the delay, whereas the pitch of IIRN[-] for the same delay is an octave lower. Chinchillas (Chinchilla laniger) were trained to discriminate IIRN[+] with a 4-ms delay from IIRN[+] with a 2-ms delay and then tested in a stimulus generalization paradigm with IIRN[+] at delays between 2 and 4 ms. Systematic gradients in behavioral response occurred along the dimension of delay, suggesting that a perceptual dimension corresponding to pitch exists for IIRN[+]. Behavioral responses to IIRN[-] test stimuli were more variable among chinchillas, suggesting that IIRN[-] did not evoke similar pitches relative to IIRN[+]. Systematic gradients in behavioral response were observed when IIRN[-] test stimuli were presented in the context of other IIRN[-] stimuli. Thus, other perceptual cues such as timbre may dominate the pitch cues when IIRN[-] test stimuli are presented in the context of IIRN[+] stimuli.     

Categorizing sounds by pitch: effects of stimulus similarity and response repetition

Two experiments were performed to determine whether categorization of the pitch of a probe tone is influenced by the pitch of, and response made to, a preceding prime tone. The prime and the probe could be drawn either from a pool of low-frequency sounds or from a pool of high-frequency sounds. The results of both experiments indicated that the performance obtained was best when the prime and the probe were the same pitch (and therefore required the same response), intermediate when the two sounds differed in pitch and required different responses, and slowest when the prime and the probe differed in pitch but required the same response (i.e., they were drawn from the same frequency pool). The results of Experiment 2 revealed in addition that when a repeated response was required, performance declined as the magnitude of the frequency change increased and that responses were made more quickly and accurately if the direction of the frequency change was away from the alternative category than if it was toward the alternative category. The results demonstrate that categorization of sounds by pitch is accomplished with reference to a previous processing episode.