Ability in perceiving nonnative contrasts: Performance on natural and synthetic speech stimuli

The perception of the distinction between /r/ and /l/ by native speakers of American English and of Japanese was studied using natural and synthetic speech. The American subjects were all nearly perfect at recognizing the natural speech sounds, whereas there was substantial variation among the Japanese subjects in their accuracy of recognizing /r/ and /l/ except in syllable-final position. A logit model, which additively combined the acoustic information conveyed byF1-transition duration and byF3-onset frequency, provided a good fit to the perception of synthetic /r/ and /l/ by the American subjects. There was substantial variation among the Japanese subjects in whether theF1 andF3 cues had a significant effect on their classifications of the synthetic speech. This variation was related to variation in accuracy of recognizing natural /r/ and /l/, such that greater use of both theF1 cue and theF3 cue in classifying the synthetic speech sounds was positively related to accuracy in recognizing the natural sounds. However, multiple regression showed that use of theF1 cue did not account for significant variance in natural speech performance beyond that accounted for by theF3 cue, indicating that theF3 cue is more important than theF1 cue for Japanese speakers learning English. The relation between performance on natural and synthetic speech also provides external validation of the logit model by showing that it predicts performance outside of the domain of data to which it was fit.     

An effect of linguistic experience: The discrimination of [r] and [l] by native speakers of Japanese and English

To test the effect of linguistic experience on the perception of a cue that is known to be effective in distinguishing between [r] and [l] in English, 21 Japanese and 39 American adults were tested on discrimination of a set of synthetic speech-like stimuli. The 13 “speech” stimuli in this set varied in the initial stationary frequency of the third formant (F3) and its subsequent transition into the vowel over a range sufficient to produce the perception of [r a] and [l a] for American subjects and to produce [r a] (which is not in phonemic contrast to [l a ]) for Japanese subjects. Discrimination tests of a comparable set of stimuli consisting of the isolated F3 components provided a “nonspeech” control. For Americans, the discrimination of the speech stimuli was nearly categorical, i.e., comparison pairs which were identified as different phonemes were discriminated with high accuracy, while pairs which were identified as the same phoneme were discriminated relatively poorly. In comparison, discrimination of speech stimuli by Japanese subjects was only slightly better than chance for all comparison pairs. Performance on nonspeech stimuli, however, was virtually identical for Japanese and American subjects; both groups showed highly accurate discrimination of all comparison pairs. These results suggest that the effect of linguistic experience is specific to perception in the “speech mode.”     

The effects of experimental variables on the perception of American English /r/ and /l/ by Japanese listeners.

The effects of variations in response categories, subjects' perception of natural speech, and stimulus range on the identification of American English /r/ and /l/ by native speakers of Japanese were investigated. Three experiments using a synthesized /rait/-/lait/ series showed that all these variables affected identification and discrimination performance by Japanese subjects. Furthermore, some of the perceptual characteristics of /r/ and /l/ for Japanese listeners were clarified: (1) Japanese listeners identified some of the stimuli of the series as /w/. (2) A positive correlation between the perception of synthesized stimuli and naturally spoken stimuli was found. Japanese listeners who were able to easily identify naturally spoken stimuli perceived the synthetic series categorically but still perceived a /w/ category on the series. (3) The stimulus range showed a striking effect on identification consistency; identification of /r/ and /l/ was strongly affected by the stimulus range, the /w/ identification less so. This indicates that Japanese listeners tend to make relative judgments between /r/ and /l/.     

The effects of experimental variables on the perception of American English /r/ and /l/ by Japanese listeners

The effects of variations in response categories, subjects’ perception of natural speech, and stimulus range on the identification of American English /r/ and /l/ by native speakers of Japanese were investigated. Three experiments using a synthesized /rait/-/lait/ series showed that all these variables affected identification and discrimination performance by Japanese-subjects. Furthermore, some of the perceptual characteristics of /r/ and /l/ for Japanese listeners were clarified: (1) Japanese listeners identified some of the stimuli of the series-as/w/.(2). Apositive correlation between the perception of synthesized stimuli and naturally-spoken stimuli was found. Japanese listeners who were able to easily identify naturally spoken stimuli perceived the synthetic series categorically but still perceived a /w/ category on the series. (3) The stimulus range showed a striking effect on identification consistency; identification of /r/ and /l/ was strongly affected by the stimulus range, the /w/ identification less so. This indicates that Japanese listeners tend to make relative judgments between /r/ and /l/.     

A limit on behavioral plasticity in speech perception

It is well attested that we perceive speech through the filter of our native language: a classic example is that of Japanese listeners who cannot discriminate between the American /l/ and /r/ and identify both as their own /r/ phoneme (Goto, 1971). Studies in the laboratory have shown, however, that perception of non-native speech sounds can be learned through training (Lively et al., 1994). This is consistent with neurophysiological evidence showing considerable experience-dependent plasticity in the brain at the first levels of sensory processing (Edeline and Weinberger, 1993; Merzenich and Sameshima, 1993; Weinberger, 1993; Kraus et al., 1995). Outside of the laboratory, however, the situation seems to differ: we here report a study involving Spanish-Catalan bilingual subjects who have had the best opportunities to learn a new contrast but did not do it. Our study demonstrates a striking lack of behavioral plasticity: early and extensive exposure to a second language is not sufficient to attain the ultimate phonological competence of native speakers.     

Learning foreign sounds in an alien world: videogame training improves non-native speech categorization

Although speech categories are defined by multiple acoustic dimensions, some are perceptually weighted more than others and there are residual effects of native-language weightings in non-native speech perception. Recent research on nonlinguistic sound category learning suggests that the distribution characteristics of experienced sounds influence perceptual cue weights: Increasing variability across a dimension leads listeners to rely upon it less in subsequent category learning (Holt & Lotto, 2006). The present experiment investigated the implications of this among native Japanese learning English /r/-/l/ categories. Training was accomplished using a videogame paradigm that emphasizes associations among sound categories, visual information, and players' responses to videogame characters rather than overt categorization or explicit feedback. Subjects who played the game for 2.5h across 5 days exhibited improvements in /r/-/l/ perception on par with 2-4 weeks of explicit categorization training in previous research and exhibited a shift toward more native-like perceptual cue weights.     

A perceptual interference account of acquisition difficulties for non-native phonemes

This article presents an account of how early language experience can impede the acquisition of non-native phonemes during adulthood. The hypothesis is that early language experience alters relatively low-level perceptual processing, and that these changes interfere with the formation and adaptability of higher-level linguistic representations. Supporting data are presented from an experiment that tested the perception of English /r/ and /l/ by Japanese, German, and American adults. The underlying perceptual spaces for these phonemes were mapped using multidimensional scaling and compared to native-language categorization judgments. The results demonstrate that Japanese adults are most sensitive to an acoustic cue, F2, that is irrelevant to the English /r/-/l/ categorization. German adults, in contrast, have relatively high sensitivity to more critical acoustic cues. The results show how language-specific perceptual processing can alter the relative salience of within- and between-category acoustic variation, and thereby interfere with second language acquisition.     

Identifiability of vowels and speakers from whispered syllables

In the present experiments, the effect of whisper register on speech perception was measured. We assessed listeners' abilities to identify 10 vowels in [hVd] context pronounced by 3 male and 3 female speakers in normal and whisper registers. Results showed 82% average identification accuracy in whisper mode, approximately a 10% falloff in identification accuracy from normally phonated speech. In both modes, significant confusions of [o] for [a] occurred, with some additional significant confusions occurring in whisper mode among vowels adjacent in F1/F2 space. We also assessed listeners' abilities to match whispered syllables with normally phonated ones by the same speaker. Each trial contained the matching syllable and two foils whispered by speakers of the same sex as the speaker of the target. Identification performance was significantly better than chance across subjects, speakers, and vowels, with no listener achieving better than 96% performance. Acoustic analyses measured potential cues to speaker identity independent of register.     

Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects

The “McGurk effect” demonstrates that visual (lip-read) information is used during speech perception even when it is discrepant with auditory information. While this has been established as a robust effect in subjects from Western cultures, our own earlier results had suggested that Japanese subjects use visual information much less than American subjects do (Sekiyama & Tohkura, 1993). The present study examined whether Chinese subjects would also show a reduced McGurk effect due to their cultural similarities with the Japanese. The subjects were 14 native speakers of Chinese living in Japan. Stimuli consisted of 10 syllables (/ba/, /pa/, /ma/, /wa/, /da/, /ta/, /na/, /ga/, /ka/, /ra/ ) pronounced by two speakers, one Japanese and one American. Each auditory syllable was dubbed onto every visual syllable within one speaker, resulting in 100 audiovisual stimuli in each language. The subjects’ main task was to report what they thought they had heard while looking at and listening to the speaker while the stimuli were being uttered. Compared with previous results obtained with American subjects, the Chinese subjects showed a weaker McGurk effect. The results also showed that the magnitude of the McGurk effect depends on the length of time the Chinese subjects had lived in Japan. Factors that foster and alter the Chinese subjects’ reliance on auditory information are discussed.     

Differences in American English, Spanish, and monkey perception of the say-stay trading relation

An interesting phenomenon in human speech perception is the trading relation, in which two different acoustic cues both signal the same phonetic percept. The present study compared American English, Spanish, and monkey listeners in their perception of the trading relation between gap duration and F1 transition onset frequency in a synthetic say-stay continuum. For all the subjects, increased gap duration caused perception to change from say to stay; however, subjects differed in the extent to which the F1 cue traded with gap duration. For American English listeners, a change from a low to a high F1 onset caused a phoneme boundary shift of 26 msec toward shorter gap durations, indicating a strong trading relation. For Spanish listeners, the shift was significantly smaller at 13.7 msec, indicating a weaker trading relation. For monkeys, there was no shift at all, indicating no trading relation. These results provide evidence that the say-stay trading relation is dependent on perceptual learning from linguistic exposure.     

Mandarin speech perception by ear and eye follows a universal principle

In this study, the nature of speech perception of native Mandarin Chinese was compared with that of American English speakers, using synthetic visual and auditory continua (from /ba/ to /da/) in an expanded factorial design. In Experiment 1, speakers identified synthetic unimodal and bimodal speech syllables as either /ba/ or /da/. In Experiment 2, Mandarin speakers were given nine possible response alternatives. Syllable identification was influenced by both visual and auditory sources of information for both Mandarin and English speakers. Performance was better described by the fuzzy logical model of perception than by an auditory dominance model or a weighted-averaging model. Overall, the results are consistent with the idea that although there may be differences in information (which reflect differences in phonemic repertoires, phonetic realizations of the syllables, and the phonotactic constraints of languages), the underlying nature of audiovisual speech processing is similar across languages.     

The perception of /r/ and /l/ in syllable-initial and syllable-final position

American English liquids /r/ and /l/ have been considered intermediate between stop consonants and vowels acoustically, articulatorily, phonologically, and perceptually. Cutting (1947a) found position-dependent ear advantages for liquids in a dichotic listening task: syllable-initial liquids produced significant right ear advantages, while syllable-final liquids produced no reliable ear advantages. The present study employed identification and discrimination tasks to determine whether /r/and /l/ are perceived differently depending on syllable position when perception is tested by a different method. Fifteen subjects listened to two synthetically produced speech series—/li/ to /ri/ and /il/ to /ir/—in which stepwise variations of the third formant cued the difference in consonant identity. The results indicated that: (1) perception did not differ between syllable positions (in contrast to the dichotic listening results), (2) liquids in both syllable positions were perceived categorically, and (3) discrimination of a nonspeech control series did not account for the perception of the speech sounds.     

English Vowel Spaces Produced by Japanese Speakers: The Smaller Point Vowels’ and the Greater Schwas’

This study investigated how Japanese-speaking learners of English pronounce the three point vowels /i/, /u/, and /a/ appearing in the first and second monosyllabic words of English noun phrases, and the schwa /ə/ appearing in English disyllabic words. First and second formant (F1 and F2) values were measured for four Japanese speakers and two American English speakers. The hypothesis that the area encompassed by the point vowels in the F1-F2 vowel space tends to be smaller for the Japanese speakers than for the English speakers was verified. The hypothesis that the area formed by the three schwas in chicken{{chick\underline{e}n}}, spoonful{{spoonf{\underline{u}}l}}, and Tarzan{{Tarz\underline{a}n}} is greater for the Japanese speakers than for the English speakers and its related hypothesis were largely upheld. Implications for further research are briefly discussed.     

/n/-/l/音位合并现象的神经机制：来自MMN的证据

采用ERP技术,比较音位合并者与非音位合并者辨别合并音位/n/-/l/的脑电和行为反应。结果发现,非音位合并者在注意前阶段展现了显著的MMN;而音位合并者没有展现MMN,但在晚期展现了一个显著的负波。行为实验发现,相比于音位合并者,非音位合并者在分辨/n/-/l/时正确率更高,反应时更短。这些结果表明：两组被试区分/n/-/l/的能力在早期自动加工阶段就已经有了显著的差异。方言经验可能降低了音位合并者在听觉感觉加工阶段检测合并音位差异的敏感度。     

Training Japanese listeners to identify English /r/and /l/: Long-term retention of learning in perception and production

Previous work from our laboratories has shown that monolingual Japanese adults who were given intensive high-variability perceptual training improved in both perception and production of English /r/-/l/ minimal pairs. In this study, we extended those findings by investigating the long-term retention of learning in both perception and production of this difficult non-native contrast. Results showed that 3 months after completion of the perceptual training procedure, the Japanese trainees maintained their improved levels of performance on the perceptual identification task. Furthermore, perceptual evaluations by native American English listeners of the Japanese trainees’ pretest, posttest, and 3-month follow-up speech productions showed that the trainees retained their long-term improvements in the general quality, identifiability, and overall intelligibility of their English /r/-/l/ word productions. Taken together, the results provide further support for the efficacy of high-variability laboratory speech sound training procedures, and suggest an optimistic outlook for the application of such procedures for a wide range of “special populations.” nt]mis|This work was supported by NIDCD Training Grant DC-00012 and by NIDCD Research Grant DC-00111 to Indiana University.     

Training Japanese listeners to identify English /r/ and /l/: long-term retention of learning in perception and production.

Previous work from our laboratories has shown that monolingual Japanese adults who were given intensive high-variability perceptual training improved in both perception and production of English /r/-/l/ minimal pairs. In this study, we extended those findings by investigating the long-term retention of learning in both perception and production of this difficult non-native contrast. Results showed that 3 months after completion of the perceptual training procedure, the Japanese trainees maintained their improved levels of performance of the perceptual identification task. Furthermore, perceptual evaluations by native American English listeners of the Japanese trainees' pretest, posttest, and 3-month follow-up speech productions showed that the trainees retained their long-term improvements in the general quality, identifiability, and overall intelligibility of their English/r/-/l/ word productions. Taken together, the results provide further support for the efficacy of high-variability laboratory speech sound training procedures, and suggest an optimistic outlook for the application of such procedures for a wide range of "special populations."     

非言语声音影响汉语听者言语声音的知觉

采用启动范式, 以汉语听者为被试, 考察了非言语声音是否影响言语声音的知觉。实验1考察了纯音对辅音范畴连续体知觉的影响, 结果发现纯音影响到辅音范畴连续体的知觉, 表现出频谱对比效应。实验2考察了纯音和复合音对元音知觉的影响, 结果发现与元音共振峰频率一致的纯音或复合音加快了元音的识别, 表现出启动效应。两个实验一致发现非言语声音能够影响言语声音的知觉, 表明言语声音知觉也需要一个前言语的频谱特征分析阶段, 这与言语知觉听觉理论的观点一致。     

Listening back in time: Does attention to memory facilitate word-in-noise identification?

The ephemeral nature of spoken words creates a challenge for oral communications where incoming speech sounds must be processed in relation to representations of just-perceived sounds stored in short-term memory. This can be particularly taxing in noisy environments where perception of speech is often impaired or initially incorrect. Usage of prior contextual information (e.g., a semantically related word) has been shown to improve speech in noise identification. In three experiments, we demonstrate a comparable effect of a semantically related cue word placed after an energetically masked target word in improving accuracy of target-word identification. This effect persisted irrespective of cue modality (visual or auditory cue word) and, in the case of cues after the target, lasted even when the cue word was presented up to 4 seconds after the target. The results are framed in the context of an attention to memory model that seeks to explain the cognitive and neural mechanisms behind processing of items in auditory memory.     

The inferential interference effects of environmental sounds on spoken speech in Japanese and British people

Spoken speech was paired with several kinds of environmental sounds and presented dichotically to both native Japanese and British subjects to compare the direction and degree of ear advantage. Results suggest that environmental sounds interfere in a similar manner for both groups of subjects but that there are highly significant differences in the degree of ear advantage between the Japanese and British subjects which might be due to some linguistic influences.