Cross-modal speech perception in adults and infants using nonspeech auditory stimuli.

Adults and infants were tested for the capacity to detect correspondences between nonspeech sounds and real vowels. The /i/ and /a/ vowels were presented in 3 different ways: auditory speech, silent visual faces articulating the vowels, or mentally imagined vowels. The nonspeech sounds were either pure tones or 3-tone complexes that isolated a single feature of the vowel without allowing the vowel to be identified. Adults perceived an orderly relation between the nonspeech sounds and vowels. They matched high-pitched nonspeech sounds to /i/ vowels and low-pitched nonspeech sounds to /a/ vowels. In contrast, infants could not match nonspeech sounds to the visually presented vowels. Infants' detection of correspondence between auditory and visual speech appears to require the whole speech signal; with development, an isolated feature of the vowel is sufficient for detection of the cross-modal correspondence.     

Evidence for precategorical extrinsic vowel normalization

Three experiments investigated whether extrinsic vowel normalization takes place largely at a categorical or a precategorical level of processing. Traditional vowel normalization effects in categorization were replicated in Experiment 1: Vowels taken from an [?]–[ε] continuum were more often interpreted as /?/ (which has a low first formant, F ₁) when the vowels were heard in contexts that had a raised F ₁ than when the contexts had a lowered F ₁. This was established with contexts that consisted of only two syllables. These short contexts were necessary for Experiment 2, a discrimination task that encouraged listeners to focus on the perceptual properties of vowels at a precategorical level. Vowel normalization was again found: Ambiguous vowels were more easily discriminated from an endpoint [ε] than from an endpoint [?] in a high-F ₁ context, whereas the opposite was true in a low-F ₁ context. Experiment 3 measured discriminability between pairs of steps along the [?]–[ε] continuum. Contextual influences were again found, but without discrimination peaks, contrary to what was predicted from the same participants’ categorization behavior. Extrinsic vowel normalization therefore appears to be a process that takes place at least in part at a precategorical processing level.     

Context effects as auditory contrast

Three experiments are reported that collectively show that listeners perceive speech sounds as contrasting auditorily with neighboring sounds. Experiment 1 replicates the well-established finding that listeners categorize more of a [d–g] continuum as [g] after [l] than after [r]. Experiments 2 and 3 show that listeners discriminate stimuli in which the energy concentrations differ in frequency between the spectra of neighboring sounds better than those in which they do not differ. In Experiment 2, [alga–arda] pairs, in which the energy concentrations in the liquid-stop sequences are H(igh) L(ow)–LH, were more discriminable than [alda–arga] pairs, in which they are HH–LL. In Experiment 3, [da] and [ga] syllables were more easily discriminated when they were preceded by lower and higher pure tones, respectively—that is, tones that differed from the stops’ higher and lower F3 onset frequencies—than when they were preceded by H and L pure tones with similar frequencies. These discrimination results show that contrast with the target’s context exaggerates its perceived value when energy concentrations differ in frequency between the target’s spectrum and its context’s spectrum. Because contrast with its context does more that merely shift the criterion for categorizing the target, it cannot be produced by neural adaptation. The finding that nonspeech contexts exaggerate the perceived values of speech targets also rules out compensation for coarticulation by showing that their values depend on the proximal auditory qualities evoked by the stimuli’s acoustic properties, rather than the distal articulatory gestures.     

Attentional set in pure- versus mixed-lists in a dichotic listening paradigm

An experiment was designed to assess the contribution of attentional set to performance on a forced choice recognition task in dichotic listening. Subjects were randomly assigned to one of three conditions: speech sounds composed of stop consonants, emotional nonspeech sounds, or a random combination of both. In the groups exposed to a single class of stimuli (pure-list), a REA (right ear advantage) emerged for the speech sounds, and a LE (left ear advantage) for the nonspeech sounds. Under mixed conditions using both classes of stimuli, no significant ear advantage was apparent, either globally or individually for the speech and nonspeech sounds. However, performance was more accurate for the left ear on nonspeech sounds and for the right ear for speech sounds, regardless of pure versus mixed placement. The results suggest that under divided attention conditions, attentional set influences the direction of the laterality effect.     

Acoustic memory and the perception of speech

The nature of acoustic memory and its relationship to the categorizing process in speech perception is investigated in three experiments on the serial recall of lists of syllables. The first study confirms previous reports that sequences comprising the syllables, bah, dah, and gah show neither enhanced retention when presented auditorily rather than visually, nor a recency effect—both occurred with sequences in which vowel sounds differed (bee, bih, boo). This was found not to be a simple vowel-consonant difference since acoustic memory effects did occur with consonant sequences that were acoustically more discriminable (sha, ma, ga and ash, am, ag). Further experiments used the stimulus suffix effect to provide evidence of acoustic memory, and showed (1), increasing the acoustic similarity of the set grossly impairs acoustic memory effects for vowels as well as consonants, and (2) such memory effects are no greater for steady-state vowels than for continuously changing diphthongs. It is concluded that the usefulness of the information that can be retrieved from acoustic memory depends on the acoustic similarity of the items in the list rather than on their phonetic class or whether or not they have “encoded” acoustic cues. These results question whether there is any psychological evidence for “encoded” speech sounds being categorized in ways different from other speech sounds.     

Identification of argentine Spanish vowels

The acoustical spectrum of the five Spanish vowels |a, e, i, o, u| has been delimited to show the areas covered by F ₁ , F ₂ , and F ₃ and the relative distribution energy among the formants. Through the analysis of the spectral components of vowels, isolated and in consonantal context, it is possible to estimate the different weight of each formant in vowel identification. At least for isolated vowels,F ₂ andF ₃ seem to be effective for the identification of [i] and [e] while theF ₁ andF ₂ carry the weight for the identification of [o] and [u]. The cue to differentiate [a] seems to beF ₂. Spanish vowels are compared with cardinals and North American English vowels. There is no correlation with cardinal vowels while similarities are found with English vowels.     

Selective adaptation of vowels

Two experiments investigating the selective adaptation of vowels examined changes in listeners’ identification functions for the vowel continuum [i-I-∈] as a function of the adapting stimulus. In Experiment I, the adapting stimuli were [i], [I], and [∈]. Both the [i] and [∈] stimuli produced significant shifts in the neighboringand distant phonetic boundaries, whereas [I] did not result in any adaptation effects. In order to explore the phonetic nature of feature adaptation in vowels, a second experiment was conducted using the adapting stimuli [gig] and [g ∈ g], which differed acoustically from the [i] and [∈] vowels on the identification continuum. Only [gig] yielded reliable adaptation effects. The results of these experiments were interpreted as suggesting arelative rather than a stableauditory mode of feature analysis in vowels and a possibly more complex auditory feature analysis for the vowel [i].     

The relationship between spoken language and speech and nonspeech processing in children with autism: a magnetic event‐related field study

It has been proposed that language impairments in children with Autism Spectrum Disorders (ASD) stem from atypical neural processing of speech and/or nonspeech sounds. However, the strength of this proposal is compromised by the unreliable outcomes of previous studies of speech and nonspeech processing in ASD. The aim of this study was to determine whether there was an association between poor spoken language and atypical event‐related field (ERF) responses to speech and nonspeech sounds in children with ASD (n = 14) and controls (n = 18). Data from this developmental population (ages 6–14) were analysed using a novel combination of methods to maximize the reliability of our findings while taking into consideration the heterogeneity of the ASD population. The results showed that poor spoken language scores were associated with atypical left hemisphere brain responses (200 to 400 ms) to both speech and nonspeech in the ASD group. These data support the idea that some children with ASD may have an immature auditory cortex that affects their ability to process both speech and nonspeech sounds. Their poor speech processing may impair their ability to process the speech of other people, and hence reduce their ability to learn the phonology, syntax, and semantics of their native language.     

Neural processing of acoustic duration and phonological German vowel length: Time courses of evoked fields in response to speech and nonspeech signals

Recent experiments showed that the perception of vowel length by German listeners exhibits the characteristics of categorical perception. The present study sought to find the neural activity reflecting categorical vowel length and the short-long boundary by examining the processing of non-contrastive durations and categorical length using MEG. Using disyllabic words with varying /a/-durations and temporally-matched nonspeech stimuli, we found that each syllable elicited an M50/M100-complex. The M50-amplitude to the second syllable varied along the durational continuum, possibly reflecting the mapping of duration onto a rhythm representation. Categorical length was reflected by an additional response elicited when vowel duration exceeded the short-long boundary. This was interpreted to reflect the integration of an additional timing unit for long in contrast to short vowels. Unlike to speech, responses to short nonspeech durations lacked a M100 to the first and M50 to the second syllable, indicating different integration windows for speech and nonspeech signals.     

Varying acoustic-phonemic ambiguity reveals that talker normalization is obligatory in speech processing

The nondeterministic relationship between speech acoustics and abstract phonemic representations imposes a challenge for listeners to maintain perceptual constancy despite the highly variable acoustic realization of speech. Talker normalization facilitates speech processing by reducing the degrees of freedom for mapping between encountered speech and phonemic representations. While this process has been proposed to facilitate the perception of ambiguous speech sounds, it is currently unknown whether talker normalization is affected by the degree of potential ambiguity in acoustic-phonemic mapping. We explored the effects of talker normalization on speech processing in a series of speeded classification paradigms, parametrically manipulating the potential for inconsistent acoustic-phonemic relationships across talkers for both consonants and vowels. Listeners identified words with varying potential acoustic-phonemic ambiguity across talkers (e.g., beet/boat vs. boot/boat) spoken by single or mixed talkers. Auditory categorization of words was always slower when listening to mixed talkers compared to a single talker, even when there was no potential acoustic ambiguity between target sounds. Moreover, the processing cost imposed by mixed talkers was greatest when words had the most potential acoustic-phonemic overlap across talkers. Models of acoustic dissimilarity between target speech sounds did not account for the pattern of results. These results suggest (a) that talker normalization incurs the greatest processing cost when disambiguating highly confusable sounds and (b) that talker normalization appears to be an obligatory component of speech perception, taking place even when the acoustic-phonemic relationships across sounds are unambiguous.

     

Auditory and phonetic coding of the cues for speech: Discrimination of the [r-l] distinction by young infants

Infants, 2 and 3 months of age, were found to discriminte stimuli along the acoustic continuum underlying the phonetic contrast [r] vs. [l] in a nearly categorical manner. For an approximately equal acoustic difference, discrimination, as measured by recovery from satiation or familiarization, was reliably better when the two stimuli were exemplars of different phonetic categories than when they were acoustic variations of the same phonetic category. Discrimination of the same acoustic information presented in a nonspeech mode was found to be continuous, that is, determined by acoustic rather than phonetic characteristics of the stimuli. The findings were discussed with reference to the nature of the mechanisms that may determine the processing of complex acoustic signals in young infants and with reference to the role of linguistic experience on the development of speech perception at the phonetic level.

     

Ability in perceiving nonnative contrasts: Performance on natural and synthetic speech stimuli

The perception of the distinction between /r/ and /l/ by native speakers of American English and of Japanese was studied using natural and synthetic speech. The American subjects were all nearly perfect at recognizing the natural speech sounds, whereas there was substantial variation among the Japanese subjects in their accuracy of recognizing /r/ and /l/ except in syllable-final position. A logit model, which additively combined the acoustic information conveyed byF1-transition duration and byF3-onset frequency, provided a good fit to the perception of synthetic /r/ and /l/ by the American subjects. There was substantial variation among the Japanese subjects in whether theF1 andF3 cues had a significant effect on their classifications of the synthetic speech. This variation was related to variation in accuracy of recognizing natural /r/ and /l/, such that greater use of both theF1 cue and theF3 cue in classifying the synthetic speech sounds was positively related to accuracy in recognizing the natural sounds. However, multiple regression showed that use of theF1 cue did not account for significant variance in natural speech performance beyond that accounted for by theF3 cue, indicating that theF3 cue is more important than theF1 cue for Japanese speakers learning English. The relation between performance on natural and synthetic speech also provides external validation of the logit model by showing that it predicts performance outside of the domain of data to which it was fit.     

Do children with autism ‘switch off’ to speech sounds? An investigation using event-related potentials

Autism is a disorder characterized by a core impairment in social behaviour. A prominent component of this social deficit is poor orienting to speech. It is unclear whether this deficit involves an impairment in allocating attention to speech sounds, or a sensory impairment in processing phonetic information. In this study, event-related potentials of 15 children with high functioning autism (mean nonverbal IQ = 109.87) and 15 typically developing children (mean nonverbal IQ = 115.73) were recorded in response to sounds in two oddball conditions. Participants heard two stimulus types: vowels and complex tones. In each condition, repetitive 'standard' sounds (condition 1: vowel; condition 2: complex tone) were replaced by a within stimulus-type 'deviant' sound and a between stimulus-type 'novel' sound. Participants' level of attention was also varied between conditions. Children with autism had significantly diminished obligatory components in response to the repetitive speech sound, but not to the repetitive nonspeech sound. This difference disappeared when participants were required to allocate attention to the sound stream. Furthermore, the children with autism showed reduced orienting to novel tones presented in a sequence of speech sounds, but not to novel speech sounds presented in a sequence of tones. These findings indicate that high functioning children with autism can allocate attention to novel speech sounds. However, they use top-down inhibition to attenuate responses to repeated streams of speech. This suggests that problems with speech processing in this population involve efferent pathways.     

Visual influences on auditory pluck and bow judgments

In the McGurk effect, visual information specifying a speaker’s articulatory movements can influence auditory judgments of speech. In the present study, we attempted to find an analogue of the McGurk effect by using nonspeech stimuli—the discrepant audiovisual tokens of plucks and bows on a cello. The results of an initial experiment revealed that subjects’ auditory judgments were influenced significantly by the visual pluck and bow stimuli. However, a second experiment in which speech syllables were used demonstrated that the visual influence on consonants was significantly greater than the visual influence observed for pluck-bow stimuli. This result could be interpreted to suggest that the nonspeech visual influence was not a true McGurk effect. In a third experiment, visual stimuli consisting of the wordspluck andbow were found to have no influence over auditory pluck and bow judgments. This result could suggest that the nonspeech effects found in Experiment 1 were based on the audio and visual information’s having an ostensive lawful relation to the specified event. These results are discussed in terms of motor-theory, ecological, and FLMP approaches to speech perception.     

An effect of linguistic experience: The discrimination of [r] and [l] by native speakers of Japanese and English

To test the effect of linguistic experience on the perception of a cue that is known to be effective in distinguishing between [r] and [l] in English, 21 Japanese and 39 American adults were tested on discrimination of a set of synthetic speech-like stimuli. The 13 “speech” stimuli in this set varied in the initial stationary frequency of the third formant (F3) and its subsequent transition into the vowel over a range sufficient to produce the perception of [r a] and [l a] for American subjects and to produce [r a] (which is not in phonemic contrast to [l a ]) for Japanese subjects. Discrimination tests of a comparable set of stimuli consisting of the isolated F3 components provided a “nonspeech” control. For Americans, the discrimination of the speech stimuli was nearly categorical, i.e., comparison pairs which were identified as different phonemes were discriminated with high accuracy, while pairs which were identified as the same phoneme were discriminated relatively poorly. In comparison, discrimination of speech stimuli by Japanese subjects was only slightly better than chance for all comparison pairs. Performance on nonspeech stimuli, however, was virtually identical for Japanese and American subjects; both groups showed highly accurate discrimination of all comparison pairs. These results suggest that the effect of linguistic experience is specific to perception in the “speech mode.”     

非言语声音影响汉语听者言语声音的知觉

采用启动范式, 以汉语听者为被试, 考察了非言语声音是否影响言语声音的知觉。实验1考察了纯音对辅音范畴连续体知觉的影响, 结果发现纯音影响到辅音范畴连续体的知觉, 表现出频谱对比效应。实验2考察了纯音和复合音对元音知觉的影响, 结果发现与元音共振峰频率一致的纯音或复合音加快了元音的识别, 表现出启动效应。两个实验一致发现非言语声音能够影响言语声音的知觉, 表明言语声音知觉也需要一个前言语的频谱特征分析阶段, 这与言语知觉听觉理论的观点一致。     

Spectral versus temporal features in dichotic listening

Ear advantage for the processing of dichotic speech sounds can be separated into two components. One of these components is an ear advantage for those phonetic features that are based on spectral acoustic cues. This ear advantage follows the direction of a given individual's ear dominance for the processing of spectral information in dichotic sounds, whether speech or nonspeech. The other factor represents a right-ear advantage for the processing of temporal information in dichotic sounds, whether speech or nonspeech. The present experiments were successful in dissociating these two factors. Since the results clearly show that ear advantage for speech is influenced by ear dominance for spectral information, a full understanding of the asymmetry in the perceptual salience of speech sounds in any individual will not be possible without knowing his ear dominance.     

Speech perception and memory coding in relation to reading ability

Previous work has demonstrated that children who are poor readers have short-term memory deficits in tasks in which the stimuli lend themselves to phonetic coding. The aim of the present study was to explore whether the poor readers' memory dificit may have its origin in perception with the encoding of the stimuli. Three experiments were conducted with third grade good and poor readers. As in earlier experiments, the poor readers were found to perform less well on recall of random word strings and to be less affected by the phonetic characteristics (rhyming or not rhyming) of the items (Experiment 1). In addition, the poor readers produced more errors of transposition (in the nonrhyming strings) than did the good readers, a further indication of the poor readers' problems with memory for order. The subjects were tested on two auditory perception tasks, one employing words (Experiment 2) and the other nonspeech environmental sounds (Experiment 3). Each was presented under two conditions: with a favorable signal-to-noise ratio and with masking. The poor readers made significantly more errors than the good readers when listening to speech in noise, but did not differ in perception of speech without noise or in perception of nonspeech environmental sounds, whether noise-masked or not. Together, the results of the perception studies suggest that poor readers have a perceptual difficulty that is specific to speech. It is suggested that the short-term memory deficits characteristic of poor readers may stem from material-specific problems of perceptual processing.     

Does irrelevant music cause an irrelevant sound effect for auditory items?

The irrelevant sound effect (ISE) describes the significant reduction in verbal serial recall during irrelevant sounds with distinct temporal-spectral variations (changing-state sound). Whereas the ISE is well-documented for the serial recall of visual items accompanied by irrelevant speech and nonspeech sounds, an ISE caused by nonspeech sounds has not been reported for auditory items. Closing this empirical gap, Experiment 1 (n=90) verified that instrumental staccato-music reduces auditory serial recall compared to legato-music and silence. Its detrimental impact was not due to perceptual masking, disturbed encoding, or increased listening effort, as the employed experimental design and methods ensured. The found nonspeech ISE in auditory serial recall is corroborated by Experiment 1b (n=60), which, by using the same experimental design and methods, replicated the well-known ISE during irrelevant changing-state speech compared to steady-state speech, pink noise, and silence.     

Feature analyzers for the phonetic dimensionstop vs. continuant

Perception of sounds along the phonetic dimensionstop vs. continuant was studied by means of a selective adaptation procedure. Subjects first identified a series of synthetic consonant-vowel syllables whose formant transitions varied in duration, slope, and amplitude characteristics. They were perceived as either [ba] or [wa]. After the initial identification test, an adapting stimulus was presented repeatedly, and then the subjects again identified the original test series. Adapting with a stop (either [ba] or [da]) led to a decrease in the number of test stimuli identified as [ba], whereas adapting with the continuant sound [wa] led to an increase in the number of [ba] identification responses. Removing the vowel portion of an adapting stimulus greatly reduced the identification shift only when the resulting stimulus was no longer perceived as speech-like. A reduction in the number of [ba] identifications occurred even when a nonspeech “stop” (the sound of a plucked string) was used as the adapting stimulus, suggesting that phonetic processing is not a necessary condition for an adaptation effect.