Effects of similarity in bandwidth on the auditory sequential streaming of two-tone complexes

We investigated the perceptual grouping of sequentially presented sounds--auditory stream segregation. It is well established that sounds heard as more similar in quality, or timbre, are more likely to be grouped into the same auditory stream. However, it is often unclear exactly what acoustic factors determine timbre. In this study, we presented various sequences of simple sounds, each comprising two frequency components (two-tone complexes), and measured their perceptual grouping. We varied only one parameter between trials, the intercomponent separation for some of the complexes, and examined the effects on stream segregation. Four hypotheses are presented that might predict the extent of streaming. Specifically, least streaming might be expected when the sounds were most similar in either (1) the frequency regions in which they have energy (maximum spectral overlap), (2) their auditory bandwidths, (3) their relative bandwidths, or (4) the rate at which the two components beat together (intermodulation rate). It was found that least streaming occurred when sounds were most similar in either their auditory or their relative bandwidths. Although these two hypotheses could not be distinguished, the results were clearly different from those predicted by hypotheses (1) and (4). The implications for models of stream segregation are discussed.     

How the brain separates sounds

In everyday life we often listen to one sound, such as someone's voice, in a background of competing sounds. To do this, we must assign simultaneously occurring frequency components to the correct source, and organize sounds appropriately over time. The physical cues that we exploit to do so are well-established; more recent research has focussed on the underlying neural bases, where most progress has been made in the study of a form of sequential organization known as "auditory streaming". Listeners' sensitivity to streaming cues can be captured in the responses of neurons in the primary auditory cortex, and in EEG wave components with a short latency (< 200ms). However, streaming can be strongly affected by attention, suggesting that this early processing either receives input from non-auditory areas, or feeds into processes that do.     

From sensory to long-term memory: evidence from auditory memory reactivation studies

Everyday experience tells us that some types of auditory sensory information are retained for long periods of time. For example, we are able to recognize friends by their voice alone or identify the source of familiar noises even years after we last heard the sounds. It is thus somewhat surprising that the results of most studies of auditory sensory memory show that acoustic details, such as the pitch of a tone, fade from memory in ca. 10-15 s. One should, therefore, ask (1) what types of acoustic information can be retained for a longer term, (2) what circumstances allow or help the formation of durable memory records for acoustic details, and (3) how such memory records can be accessed. The present review discusses the results of experiments that used a model of auditory recognition, the auditory memory reactivation paradigm. Results obtained with this paradigm suggest that the brain stores features of individual sounds embedded within representations of acoustic regularities that have been detected for the sound patterns and sequences in which the sounds appeared. Thus, sounds closely linked with their auditory context are more likely to be remembered. The representations of acoustic regularities are automatically activated by matching sounds, enabling object recognition.     

Infant crying: Acoustics,perception and communication

This paper first reviews briefly the literature on the acoustics of infant cry sounds and then presents two empirical studies on the perception of cry and noncry sounds in their social-communicative context. Acoustic analysis of cry sounds has undergone dramatic changes in the last 35 years, including the introduction of more than a hundred different acoustic measures. The study of cry acoustics, however, remains largely focused on neonates who have various medical problems or are at risk for developmental delays. Relatively little is known about how cry sounds and cry perception change developmentally, or about how they compare with noncry sounds. The data presented here support the notion that both auditory and visual information are important in caregivers' interpretations of infant sounds in naturalistic contexts. When only auditory information is available (Study 1), cry sounds become generally more recognizable from 3 to 12 months of age; perception of noncry sounds, however, generally does not change over age. When auditory and visual information contradict each other (Study 2), adults tend to perform at chance levels, with a few interesting exceptions. It is suggested that broadening studies of acoustic analysis and perception to include both cry and noncry sounds should increase our understanding of the development of communication in infancy. Finally, we suggest that examining the cry in its developmental context holds great possibility for delineating the factors that underlie adults' responses to crying.     

The effects of auditory streaming on duplex perception

When a formant transition and the remainder of a syllable are presented to subjects' opposite ears, most subjects perceive two simultaneous sounds: a syllable and a nonspeech chirp. It has been demonstrated that, when the remainder of the syllable (base) is kept unchanged, the identity of the perceived syllable will depend on the kind of transition presented at the opposite ear. This phenomenon, called duplex perception, has been interpreted as the result of the independent operation of two perceptual systems or modes, the phonetic and the auditory mode. In the present experiments, listeners were required to identify and discriminate such duplex syllables. In some conditions, the isolated transition was embedded in a temporal sequence of capturing transitions sent to the same ear. This streaming procedure significantly weakened the contribution of the transition to the perceived phonetic identity of the syllable. It is likely that the sequential integration of the isolated transition into a sequence of capturing transitions affected its fusion with the contralateral base. This finding contrasts with the idea that the auditory and phonetic processes are operating independently of each other. The capturing effect seems to be more consistent with the hypothesis that duplex perception occurs in the presence of conflicting cues for the segregation and the integration of the isolated transition with the base.     

Structure and function of auditory cortex: music and speech

We examine the evidence that speech and musical sounds exploit different acoustic cues: speech is highly dependent on rapidly changing broadband sounds, whereas tonal patterns tend to be slower, although small and precise changes in frequency are important. We argue that the auditory cortices in the two hemispheres are relatively specialized, such that temporal resolution is better in left auditory cortical areas and spectral resolution is better in right auditory cortical areas. We propose that cortical asymmetries might have developed as a general solution to the need to optimize processing of the acoustic environment in both temporal and frequency domains.     

Effects of Action Video Game Experience on Change Detection

Reflected sounds are often treated as an acoustic problem because they produce false localization cues and decrease speech intelligibility. However, their properties are shaped by the acoustic properties of the environment and therefore are a potential source of information about that environment. The objective of this study was to determine whether information carried by reflected sounds can be used by listeners to enhance their awareness of their auditory environment. Twelve listeners participated in two auditory training tasks in which they learned to identify three environments based on a limited subset of sounds and then were tested to determine whether they could transfer that learning to new, unfamiliar sounds. Results showed that significant learning occurred despite the task difficulty. An analysis of stimulus attributes suggests that it is easiest to learn to identify reflected sound when it occurs in sounds with longer decay times and broadly distributed dominant spectral components.     

Superior formation of cortical memory traces for melodic patterns in musicians

The human central auditory system has a remarkable ability to establish memory traces for invariant features in the acoustic environment despite continual acoustic variations in the sounds heard. By recording the memory-related mismatch negativity (MMN) component of the auditory electric and magnetic brain responses as well as behavioral performance, we investigated how subjects learn to discriminate changes in a melodic pattern presented at several frequency levels. In addition, we explored whether musical expertise facilitates this learning. Our data show that especially musicians who perform music primarily without a score learn easily to detect contour changes in a melodic pattern presented at variable frequency levels. After learning, their auditory cortex detects these changes even when their attention is directed away from the sounds. The present results thus show that, after perceptual learning during attentive listening has taken place, changes in a highly complex auditory pattern can be detected automatically by the human auditory cortex and, further, that this process is facilitated by musical expertise.     

Perceptual organization of complex auditory sequences: effect of number of simultaneous subsequences and frequency separation

Previous findings on streaming are generalized to sequences composed of more than 2 subsequences. A new paradigm identified whether listeners perceive complex sequences as a single unit (integrative listening) or segregate them into 2 (or more) perceptual units (stream segregation). Listeners heard 2 complex sequences, each composed of 1, 2, 3, or 4 subsequences. Their task was to detect a temporal irregularity within 1 subsequence. In Experiment 1, the smallest frequency separation under which listeners were able to focus on 1 subsequence was unaffected by the number of co-occurring subsequences; nonfocused sounds were not perceptually organized into streams. In Experiment 2, detection improved progressively, not abruptly, as the frequency separation between subsequences increased from 0.25 to 6 auditory filters. The authors propose a model of perceptual organization of complex auditory sequences.     

Effects of attention and unilateral neglect on auditory stream segregation

Two pairs of experiments studied the effects of attention and of unilateral neglect on auditory streaming. The first pair showed that the build up of auditory streaming in normal participants is greatly reduced or absent when they attend to a competing task in the contralateral ear. It was concluded that the effective build up of streaming depends on attention. The second pair showed that patients with an attentional deficit toward the left side of space (unilateral neglect) show less stream segregation of tone sequences presented to their left than to their right ears. Streaming in their right ears was similar to that for stimuli presented to either ear of healthy and of brain-damaged controls, who showed no across-ear asymmetry. This result is consistent with an effect of attention on streaming, constrains the neural sites involved, and reveals a qualitative difference between the perception of left- and right-sided sounds by neglect patients.     

What Are You Waiting For? Real‐Time Integration of Cues for Fricatives Suggests Encapsulated Auditory Memory

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of early cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing.     

没听到？“听”到！——来自听觉表象神经机制研究的证据

近年来听觉表象开始得到关注,相关研究包括言语声音、音乐声音、环境声音的听觉表象三类。本文梳理了认知神经科学领域对上述三种听觉表象所激活的脑区研究,比较了听觉表象和听觉对应脑区的异同,并展望了听觉表象未来的研究方向。     

Grammatical language impairment and the specificity of cognitive domains: relations between auditory and language abilities

Grammatical-specific language impairment (G-SLI) in children, arguably, provides evidence for the existence of a specialised grammatical sub-system in the brain, necessary for normal language development. Some researchers challenge this, claiming that domain-general, low-level auditory deficits, particular to rapid processing, cause phonological deficits and thereby SLI. We investigate this possibility by testing the auditory discrimination abilities of G-SLI children for speech and non-speech sounds, at varying presentation rates, and controlling for the effects of age and language on performance. For non-speech formant transitions, 69% of the G-SLI children showed normal auditory processing, whereas for the same acoustic information in speech, only 31% did so. For rapidly presented tones, 46% of the G-SLI children performed normally. Auditory performance with speech and non-speech sounds differentiated the G-SLI children from their age-matched controls, whereas speed of processing did not. The G-SLI children evinced no relationship between their auditory and phonological/grammatical abilities. We found no consistent evidence that a deficit in processing rapid acoustic information causes or maintains G-SLI. The findings, from at least those G-SLI children who do not exhibit any auditory deficits, provide further evidence supporting the existence of a primary domain-specific deficit underlying G-SLI.     

Auditory color constancy: Calibration to reliable spectral properties across nonspeech context and targets

Brief experience with reliable spectral characteristics of a listening context can markedly alter perception of subsequent speech sounds, and parallels have been drawn between auditory compensation for listening context and visual color constancy. In order to better evaluate such an analogy, the generality of acoustic context effects for sounds with spectral-temporal compositions distinct from speech was investigated. Listeners identified nonspeech sounds—extensively edited samples produced by a French horn and a tenor saxophone—following either resynthesized speech or a short passage of music. Preceding contexts were “colored” by spectral envelope difference filters, which were created to emphasize differences between French horn and saxophone spectra. Listeners were more likely to report hearing a saxophone when the stimulus followed a context filtered to emphasize spectral characteristics of the French horn, and vice versa. Despite clear changes in apparent acoustic source, the auditory system calibrated to relatively predictable spectral characteristics of filtered context, differentially affecting perception of subsequent target nonspeech sounds. This calibration to listening context and relative indifference to acoustic sources operates much like visual color constancy, for which reliable properties of the spectrum of illumination are factored out of perception of color.     

Auditory grouping

Although our subjective experience of the world is one of discrete sound sources, the individual frequency components that make up these separate sources are spread across the frequency spectrum. Listeners. use various simple cues, including common onset time and harmonicity, to help them achieve this perceptual separation. Our ability to use harmonicity to segregate two simultaneous sound sources is constrained by the frequency resolution of the auditory system, and is much more effective for low-numbered, resolved harmonics than for higher-numbered, unresolved ones. Our ability to use interaural time-differences (ITDs) in perceptual segregation poses a paradox. Although ITDs are the dominant cue for the localization of complex sounds, listeners cannot use ITDs alone to segregate the speech of a single talker from similar simultaneous sounds. Listeners are, however, very good at using ITD to track a particular sound source across time. This difference might reflect two different levels of auditory processing, indicating that listeners attend to grouped auditory objects rather than to those frequencies that share a common ITD.     

Seeing to hear better: evidence for early audio-visual interactions in speech identification

Lip reading is the ability to partially understand speech by looking at the speaker's lips. It improves the intelligibility of speech in noise when audio-visual perception is compared with audio-only perception. A recent set of experiments showed that seeing the speaker's lips also enhances sensitivity to acoustic information, decreasing the auditory detection threshold of speech embedded in noise [J. Acoust. Soc. Am. 109 (2001) 2272; J. Acoust. Soc. Am. 108 (2000) 1197]. However, detection is different from comprehension, and it remains to be seen whether improved sensitivity also results in an intelligibility gain in audio-visual speech perception. In this work, we use an original paradigm to show that seeing the speaker's lips enables the listener to hear better and hence to understand better. The audio-visual stimuli used here could not be differentiated by lip reading per se since they contained exactly the same lip gesture matched with different compatible speech sounds. Nevertheless, the noise-masked stimuli were more intelligible in the audio-visual condition than in the audio-only condition due to the contribution of visual information to the extraction of acoustic cues. Replacing the lip gesture by a non-speech visual input with exactly the same time course, providing the same temporal cues for extraction, removed the intelligibility benefit. This early contribution to audio-visual speech identification is discussed in relationships with recent neurophysiological data on audio-visual perception.     

Electrophysiologic Manifestations of Impaired Temporal Lobe Auditory Processing in Verbal Auditory Agnosia

The present study examined the extent to which verbal auditory agnosia (VAA) is primarily a phonemic decoding disorder, as contrasted to a more global defect in acoustic processing. Subjects were six young adults who presented with VAA in childhood and who, at the time of testing, showed varying degrees of residual auditory discrimination impairment. They were compared to a group of young adults with normal language development matched for age and gender. Cortical event-related potentials (ERPs) were recorded to tones and to consonant-vowel stimuli presented in an "oddball" discrimination paradigm. In addition to cortical ERPs, auditory brainstem responses (ABRs) and middle latency responses (MLRs) were recorded. Cognitive and language assessments were obtained for the VAA subjects. ABRs and MLRs were normal. In comparison with the control group, the cortical ERPs of the VAA subjects showed a delay in the N1 component recorded over lateral temporal cortex both to tones and to speech sounds, despite an N1 of normal latency overlying the frontocentral region of the scalp. These electrophysiologic findings indicate a slowing of processing of both speech and nonspeech auditory stimuli and suggest that the locus of this abnormality is within the secondary auditory cortex in the lateral surface of the temporal lobes.     

Preliminary evidence for selective cortical responses to music in one-month-old infants

Prior studies have observed selective neural responses in the adult human auditory cortex to music and speech that cannot be explained by the differing lower-level acoustic properties of these stimuli. Does infant cortex exhibit similarly selective responses to music and speech shortly after birth? To answer this question, we attempted to collect functional magnetic resonance imaging (fMRI) data from 45 sleeping infants (2.0- to 11.9-weeks-old) while they listened to monophonic instrumental lullabies and infant-directed speech produced by a mother. To match acoustic variation between music and speech sounds we (1) recorded music from instruments that had a similar spectral range as female infant-directed speech, (2) used a novel excitation-matching algorithm to match the cochleagrams of music and speech stimuli, and (3) synthesized “model-matched” stimuli that were matched in spectrotemporal modulation statistics to (yet perceptually distinct from) music or speech. Of the 36 infants we collected usable data from, 19 had significant activations to sounds overall compared to scanner noise. From these infants, we observed a set of voxels in non-primary auditory cortex (NPAC) but not in Heschl's Gyrus that responded significantly more to music than to each of the other three stimulus types (but not significantly more strongly than to the background scanner noise). In contrast, our planned analyses did not reveal voxels in NPAC that responded more to speech than to model-matched speech, although other unplanned analyses did. These preliminary findings suggest that music selectivity arises within the first month of life. A video abstract of this article can be viewed at https://youtu.be/c8IGFvzxudk .

Research Highlights

• Responses to music, speech, and control sounds matched for the spectrotemporal modulation-statistics of each sound were measured from 2- to 11-week-old sleeping infants using fMRI.
• Auditory cortex was significantly activated by these stimuli in 19 out of 36 sleeping infants.
• Selective responses to music compared to the three other stimulus classes were found in non-primary auditory cortex but not in nearby Heschl's Gyrus.
• Selective responses to speech were not observed in planned analyses but were observed in unplanned, exploratory analyses.

     

早期听觉剥夺后的大脑可塑性：来自先天性听力障碍群体的证据

皮层功能的正常发展依赖于充分的外部感觉信息的输入。先天性听力障碍群体由于经历早期听觉剥夺, 皮层功能往往出现异常。具体表现为初级听皮层功能退化, 初级、次级听皮层的功能连接变弱, 次级听皮层出现跨通道功能重组; 在后天听力重建后听皮层功能重组仍然存在, 言语加工需要更多高级认知资源的补偿。已有研究在探讨听力重建后皮层的长期可塑性机制、复杂声学环境下言语加工机制、汉语言加工独特性等方面尚不深入, 值得进一步研究。     

Comparing simulator sickness in younger and older adults during simulated driving under different multisensory conditions

Driving simulators are valuable tools for traffic safety research as they allow for systematic reproductions of challenging situations that cannot be easily tested during real-world driving. Unfortunately, simulator sickness (i.e., nausea, dizziness, etc.) is common in many driving simulators and may limit their utility. The experience of simulator sickness is thought to be related to the sensory feedback provided to the user and is also thought to be greater in older compared to younger users. Therefore, the present study investigated whether adding auditory and/or motion cues to visual inputs in a driving simulator affected simulator sickness in younger and older adults. Fifty-eight healthy younger adults (age 18–39) and 63 healthy older adults (age 65+) performed a series of simulated drives under one of four sensory conditions: (1) visual cues alone, (2) combined visual + auditory cues (engine, tire, wind sounds), (3) combined visual + motion cues (via hydraulic hexapod motion platform), or (4) a combination of all three sensory cues (visual, auditory, motion). Simulator sickness was continuously recorded while driving and up to 15 min after driving session termination. Results indicated that older adults experienced more simulator sickness than younger adults overall and that females were more likely to drop out and drove for less time compared to males. No differences between sensory conditions were observed. However, older adults needed significantly longer time to fully recover from the driving session than younger adults, particularly in the visual-only condition. Participants reported that driving in the simulator was least realistic in the visual-only condition compared to the other conditions. Our results indicate that adding auditory and/or motion cues to the visual stimulus does not guarantee a reduction of simulator sickness per se, but might accelerate the recovery process, particularly in older adults.