Cross-modal speech perception in adults and infants using nonspeech auditory stimuli.

Adults and infants were tested for the capacity to detect correspondences between nonspeech sounds and real vowels. The /i/ and /a/ vowels were presented in 3 different ways: auditory speech, silent visual faces articulating the vowels, or mentally imagined vowels. The nonspeech sounds were either pure tones or 3-tone complexes that isolated a single feature of the vowel without allowing the vowel to be identified. Adults perceived an orderly relation between the nonspeech sounds and vowels. They matched high-pitched nonspeech sounds to /i/ vowels and low-pitched nonspeech sounds to /a/ vowels. In contrast, infants could not match nonspeech sounds to the visually presented vowels. Infants' detection of correspondence between auditory and visual speech appears to require the whole speech signal; with development, an isolated feature of the vowel is sufficient for detection of the cross-modal correspondence.     

Categorical results do not imply categorical perception

Categorical perception refers to the ability to discriminate between- but not within-category differences along a stimulus continuum. Although categorical perception was thought to be unique to speech, recent studies have yielded similar results with nonspeech continua. The results are usually interpreted in terms of categorical, as opposed to continuous, perception of both speech and nonspeech continua. In contrast, we argue that these continua are perceived continuously, although they are characterized by relatively large increases in discrim-inability near the category boundary. To support this argument, the amplitude rise time of a tone was varied to produce either an increase or a decrease in the intensity during the initial portion of the tone. A bipolar continuum of onset times increasing and decreasing in amplitude yielded traditional categorical results. However, when only half of this continuum was tested, subjects perceived the same sounds continuously. The finding of traditional categorical results along the bipolar continuum, when the sounds were shown to be perceived continuously in another context, argues against the use of traditional categorical results as evidence for categorical perception.     

Susceptibility of a stop consonant to adaptation on a speech-nonspeech continuum: Further evidence against feature detectors in speech perception

The present experiment uses the perceptual adaptation paradigm to establish the validity of a previous test of the feature detector model of speech perception. In the present study, a synthetic stimulus series varied from a CV syllable, [ba], to a nonspeech buzz. When the endpoint tokens were employed alternatively as adaptors, the category boundary was shifted relative to unadapted identification in each adaptor condition. This result suggests that a prior test which used a vowel as the speech endpoint was legitimate because a stop consonant, an exemplary speech sound, was also susceptible to perceptual adaptation in a speech-nonspeech context. Feature detector models predict, incorrectly, that this outcome is impossible. Therefore, this finding may be taken to undermine the interpretation of adaptation as fatigue in a set of detectors tuned to detect the distinctive features of linguistic analysis.     

Cross-series adaptation using song and string

The acoustic-auditory feature “risetime” has been claimed to underlie both the phoneticaffricate-fricative distinction and the nonphoneticplucked-string/bowed-string distinction. We used the perceptual adaptation technique to determine whether the risetime differences of the [d3a]-[3a] distinction would therefore be registered by the same mechanism that mediates risetime differences for the plucked-bowed distinction. Two continua were used, one of digitally modified natural speech and one of synthetic violin sounds, in which the risetime was varied across each set of tokens from 0 msec to 80 msec in steps of 10 msec. The speech was sung and the violin notes were synthesized with the same fundamental frequency, 294 Hz. Adaptation of the category boundaries was observed only when speech adaptors were tested with the speech continuum and when violin adaptors were tested with the violin continuum. When crossseries tests were performed (violin adaptors tested with the speech series, and speech adaptors tested with the violin series), no effect of adaptation was observed. This finding indicates that these speech and violin sounds, despite obvious acoustic similarities, do not share the same feature detectors.     

Neural processing of acoustic duration and phonological German vowel length: Time courses of evoked fields in response to speech and nonspeech signals

Recent experiments showed that the perception of vowel length by German listeners exhibits the characteristics of categorical perception. The present study sought to find the neural activity reflecting categorical vowel length and the short-long boundary by examining the processing of non-contrastive durations and categorical length using MEG. Using disyllabic words with varying /a/-durations and temporally-matched nonspeech stimuli, we found that each syllable elicited an M50/M100-complex. The M50-amplitude to the second syllable varied along the durational continuum, possibly reflecting the mapping of duration onto a rhythm representation. Categorical length was reflected by an additional response elicited when vowel duration exceeded the short-long boundary. This was interpreted to reflect the integration of an additional timing unit for long in contrast to short vowels. Unlike to speech, responses to short nonspeech durations lacked a M100 to the first and M50 to the second syllable, indicating different integration windows for speech and nonspeech signals.     

Categorical perception of nonspeech chirps and bleats

Mattingly, Liberman, Syrdal, and Halwes, (1971) claimed to demonstrate that subjects cannot classify nonspeech chirp and bleat continua, but that they can classify into three categories a syllable place continuum whose variation is physically identical to the nonspeech chirp and bleat continua. This finding for F2 transitions, as well as similar findings for F3 transitions, has been cited as one source of support for theories that different modes or modules underlie the perception of speech and nonspeech acoustic stimuli. However, this pattern of finding for speech and nonspeech continua may be the result of research methods rather than a true difference in subject ability. Using tonal stimuli based on the nonspeech stimuli of Mattingly et al., we found that subjects, with appropriate practice, could classify nonspeech chirp, short bleat, and bleat continua with boundaries equivalent to the syllable place continuum of Mattingly et al. With the possible exception of the higher frequency boundary for both our bleats and the Mattingly syllables, ABX discrimination peaks were clearly present and corresponded in location to the given labeling boundary.     

非言语声音影响汉语听者言语声音的知觉

采用启动范式, 以汉语听者为被试, 考察了非言语声音是否影响言语声音的知觉。实验1考察了纯音对辅音范畴连续体知觉的影响, 结果发现纯音影响到辅音范畴连续体的知觉, 表现出频谱对比效应。实验2考察了纯音和复合音对元音知觉的影响, 结果发现与元音共振峰频率一致的纯音或复合音加快了元音的识别, 表现出启动效应。两个实验一致发现非言语声音能够影响言语声音的知觉, 表明言语声音知觉也需要一个前言语的频谱特征分析阶段, 这与言语知觉听觉理论的观点一致。     

The ABCs of categorical perception

Studies of speech perception first revealed a surprising discontinuity in the way in which stimulus values on a physical continuum are perceived. Data which demonstrate the effect in nonspeech modes have challenged the contention that categorical perception is a hallmark of the speech mode, but the psychophysical models that have been proposed have not resolved the issues raised by empirical findings. This study provides data from judgments of four sensory continua, two visual and two tactual-kinesthetic, which show that the adaptation level for a set of stimuli serves as a category boundary whether stimuli on the continuum differ by linear or logarithmic increments. For all sensory continua studied, discrimination of stimuli belonging to different perceptual categories was more accurate than discrimination of stimuli belonging to the same perceptual category. Moreover, shifts in the adaptation level produced shifts in the location of the category boundary. The concept of Adaptation-level Based Categorization (ABC) provides a unified account of judgmental processes in categorical perception without recourse to post hoc constructs such as implicit anchors or external referents.     

On the causes of compensation for coarticulation: evidence for phonological mediation

This study examined whether compensation for coarticulation in fricative-vowel syllables is phonologically mediated or a consequence of auditory processes. Smits (2001a) had shown that compensation occurs for anticipatory lip rounding in a fricative caused by a following rounded vowel in Dutch. In a first experiment, the possibility that compensation is due to general auditory processing was investigated using nonspeech sounds. These did not cause context effects akin to compensation for coarticulation, although nonspeech sounds influenced speech sound identification in an integrative fashion. In a second experiment, a possible phonological basis for compensation for coarticulation was assessed by using audiovisual speech. Visual displays, which induced the perception of a rounded vowel, also influenced compensation for anticipatory lip rounding in the fricative. These results indicate that compensation for anticipatory lip rounding in fricative-vowel syllables is phonologically mediated. This result is discussed in the light of other compensation-for-coarticulation findings and general theories of speech perception.     

Central and peripheral representation of whispered and voiced speech

Whispered speech is very different acoustically from normally voiced speech, yet listeners appear to have little trouble perceiving whispered speech. Two selective adaptation experiments explored the basis for the common perception of whispered and voiced speech, using two synthetic /ba/-/wa/ continua (one voiced, and one whispered). In the first experiment the endpoints of each series were used as adaptors, along with several nonspeech adaptors. Speech adaptors produced reliable labeling shifts of syllables matching in periodicity (i.e., whispered-whispered or voiced-voiced); somewhat smaller effects were found with mismatched periodicity. A periodic nonspeech tone with short rise time (the "pluck") produced adaptation effects like those for /ba/. These shifts occurred for whispered test syllables as well as voiced ones, indicating a common abstract level of representation for voiced and whispered stimuli. Experiment 2 replicated and extended Experiment 1, using same-ear and cross-ear adaptation conditions. There was perfect cross-ear transfer of the nonspeech adaptation effect, again implicating an abstract level of representation. The results support the existence of two levels of processing for complex acoustic signals. The commonality of whispered and voiced speech arises at the second, abstract level. Both this level, and the earlier, more directly acoustic level, are susceptible to adaptation effects.     

Speech perception in rats: use of duration and rise time cues in labeling of affricate/fricative sounds

The voiceless affricate/fricative contrast has played an important role in developing auditory theories of speech perception. This type of theory draws some of its support from experimental data on animals. However, nothing is known about differential responding of affricate/fricative continua by animals. In the current study, the ability of hooded rats to "label" an affricate/fricative continuum was tested. Transfer (without retraining) to analogous nonspeech continua was also tested. The nonspeech continua were chosen so that if transfer occurred, it would indicate whether the animals had learned to use rise time or duration cues to differentiate affricates from fricatives. The data from 9 of 10 rats indicated that rats can discriminate between these cues and do so in a similar manner to human subjects. The data from 9 of 10 rats also demonstrated that the rise time of the stimulus was the basis of the discrimination; the remaining rat appeared to use duration.     

元音范畴知觉中特征分析和整合的时间进程

采用启动范式,在三个实验中通过操纵启动音和目标音的频谱相似度和时间间隔,考察了汉语听者元音范畴知觉中特征分析和整合的时间进程。结果发现随着启动音（从纯音、复合音到目标元音本身）和目标元音频谱相似度的增加,启动效应延续的时间越来越长。实验结果支持语音范畴知觉存在早期的声学特征分析和整合到后期的范畴知觉阶段,并为这些加工阶段的时间进程提供了初步的证据。     

Spectral versus temporal features in dichotic listening

Ear advantage for the processing of dichotic speech sounds can be separated into two components. One of these components is an ear advantage for those phonetic features that are based on spectral acoustic cues. This ear advantage follows the direction of a given individual's ear dominance for the processing of spectral information in dichotic sounds, whether speech or nonspeech. The other factor represents a right-ear advantage for the processing of temporal information in dichotic sounds, whether speech or nonspeech. The present experiments were successful in dissociating these two factors. Since the results clearly show that ear advantage for speech is influenced by ear dominance for spectral information, a full understanding of the asymmetry in the perceptual salience of speech sounds in any individual will not be possible without knowing his ear dominance.     

辅音和元音知觉中的启动效应差异

采用启动范式,在两个实验中分别考察了辅音范畴和元音范畴知觉中的启动效应。启动音是纯音和目标范畴本身,目标音是辅音范畴和元音范畴连续体。结果发现辅音范畴连续体知觉的范畴反应百分比受到纯音和言语启动音影响,辅音范畴知觉的反应时只受言语启动音影响;元音范畴连续体知觉的范畴反应百分比不受两种启动音影响,但元音范畴知觉的反应时受到言语启动音影响。实验结果表明辅音范畴和元音范畴知觉中的启动效应存在差异,这为辅音和元音范畴内在加工机制的差异提供了新证据。     

Adaptation of the relative onset time of two-component tones

The results of three selective adaptation experiments employing nonspeech signals that differed in temporal onset are reported. In one experiment, adaptation effects were observed when both the adapting and test stimuli were selected from the same nonspeech test continuum. This result was interpreted as evidence for selective processing of temporal order information in nonspeech signals. Two additional experiments tested for the presence of cross-series adaptation effects from speech to nonspeech and then from nonspeech to speech. Both experiments failed to show any evidence of cross-series adaptation effects, implying a possible dissociation between perceptual classes of speech and nonspeech signals in processing temporal order information. Despite the absence of cross-series effects, it is argued that the ability of the auditory system to process temporal order information may still provide a possible basis for explaining the perception of voicing in stops that differ in VOT. The results of the present experiments, taken together with earlier findings on the perception of temporal onset in nonspeech signals, were viewed as an example of the way spoken language has exploited the basic sensory capabilities of the auditory system to signal phonetic differences.     

Talker continuity and the use of rate information during phonetic perception

Research has shown that speaking rate provides an important context for the perception of certain acoustic properties of speech. For example, syllable duration, which varies as a function of speaking rate, has been shown to influence the perception of voice onset time (VOT) for syllableinitial stop consonants. The purpose of the present experiments was to examine the influence of syllable duration when the initial portion of the syllable was produced by one talker and the remainder of the syllable was produced by a different talker. A short-duration and a long-duration /bi/-/pi/ continuum were synthesized with pitch and formant values appropriate to a female talker. When presented to listeners for identification, these stimuli demonstrated the typical effect of syllable duration on the voicing boundary: a shorter VOT boundary for the short stimuli than for the long stimuli. An /i/ vowel, synthesized with pitch and formant values appropriate to a male talker, was added to the end of each of the short tokens, producing a new hybrid continuum. Although the overall syllable duration of the hybrid stimuli equaled the original long stimuli, they produced a VOT boundary similar to that for the short stimuli. In a second experiment, two new /i/ vowels were synthesized. One had a pitch appropriate to a female talker with formant values appropriate to a male talker; the other had a pitch appropriate to a male talker and formants appropriate to a female talker. These vowels were used to create two new hybrid continua. In a third experiment, new hybrid continua were created by using more extreme male formant values. The results of both experiments demonstrated that the hybrid tokens with a change in pitch acted like the short stimuli, whereas the tokens with a change in formants acted like the long stimuli. A fourth experiment demonstrated that listeners could hear a change in talker with both sets of hybrid tokens. These results indicate that continuity of pitch but not formant structure appears to be the critical factor in the calculation of speaking rate within a syllable.     

Constraints on the processes responsible for the extrinsic normalization of vowels

Listeners tune in to talkers’ vowels through extrinsic normalization. We asked here whether this process could be based on compensation for the long-term average spectrum (LTAS) of preceding sounds and whether the mechanisms responsible for normalization are indifferent to the nature of those sounds. If so, normalization should apply to nonspeech stimuli. Previous findings were replicated with first-formant (F1) manipulations of speech. Targets on a [pt]–[p?t] (low–high F1) continuum were labeled as [pt] more after high-F1 than after low-F1 precursors. Spectrally rotated nonspeech versions of these materials produced similar normalization. None occurred, however, with nonspeech stimuli that were less speechlike, even though precursor–target LTAS relations were equivalent to those used earlier. Additional experiments investigated the roles of pitch movement, amplitude variation, formant location, and the stimuli's perceived similarity to speech. It appears that normalization is not restricted to speech but that the nature of the preceding sounds does matter. Extrinsic normalization of vowels is due, at least in part, to an auditory process that may require familiarity with the spectrotemporal characteristics of speech.     

Perceptual grouping of speech components differing in fundamental frequency and onset-time

Are there general auditory grouping principles that allow the sounds of a single speaker to be grouped together before phonetic categorisation? Four experiments are reported on the use made of a common fundamental frequency or a common starting time in grouping formants together to form phonetic categories. The first experiment shows that the perception of a vowel category is unaffected by formants being excited at different fundamentals or starting at 100-ms intervals. The second and third experiments show no effect of a different fundamental on the combination of the timbres of pairs of formants presented either binaurally or dichotically to form diphthongs. Onset-time also has no effect with binaural presentation. The fourth experiment finds both an effect of grouping formants by a common fundaental using formant trajectories that do not overlap in frequency, and also an effect of onset-time. Neither a common fundamental nor common onset-time is either a necessary or a sufficient condition for formants to be grouped into a common speech category, although they can be shown to exert an influence. Both these variables exert a considerable influence on the number of sounds that subjects report hearing, even under conditions where they do not influence the reported speech category, indicating a dissociation between mechanisms concerned with “how many” sound sources there are, and those concerned with “what” a source consists of.     

Adaptation of speech by nonspeech: evidence for complex acoustic cue detectors

Three selective adaptation experiments were run, using nonspeech stimuli (music and noise) to adapt speech continua ([ba]-[wa] and [cha]-[sha]). The adaptors caused significant phoneme boundary shifts on the speech continua only when they matched in periodicity: Music stimuli adapted [ba]-[wa], whereas noise stimuli adapted [cha]-[sha]. However, such effects occurred even when the adaptors and test continua did not match in other simple acoustic cues (rise time or consonant duration). Spectral overlap of adaptors and test items was also found to be unnecessary for adaptation. The data support the existence of auditory processors sensitive to complex acoustic cues, as well as units that respond to more abstract properties. The latter are probably at a level previously thought to be phonetic. Asymmetrical adaptation was observed, arguing against an opponent-process arrangement of these units. A two-level acoustic model of the speech perception process is offered to account for the data.     

Do children with autism ‘switch off’ to speech sounds? An investigation using event-related potentials

Autism is a disorder characterized by a core impairment in social behaviour. A prominent component of this social deficit is poor orienting to speech. It is unclear whether this deficit involves an impairment in allocating attention to speech sounds, or a sensory impairment in processing phonetic information. In this study, event-related potentials of 15 children with high functioning autism (mean nonverbal IQ = 109.87) and 15 typically developing children (mean nonverbal IQ = 115.73) were recorded in response to sounds in two oddball conditions. Participants heard two stimulus types: vowels and complex tones. In each condition, repetitive 'standard' sounds (condition 1: vowel; condition 2: complex tone) were replaced by a within stimulus-type 'deviant' sound and a between stimulus-type 'novel' sound. Participants' level of attention was also varied between conditions. Children with autism had significantly diminished obligatory components in response to the repetitive speech sound, but not to the repetitive nonspeech sound. This difference disappeared when participants were required to allocate attention to the sound stream. Furthermore, the children with autism showed reduced orienting to novel tones presented in a sequence of speech sounds, but not to novel speech sounds presented in a sequence of tones. These findings indicate that high functioning children with autism can allocate attention to novel speech sounds. However, they use top-down inhibition to attenuate responses to repeated streams of speech. This suggests that problems with speech processing in this population involve efferent pathways.