Word segmentation with universal prosodic cues

When listening to speech from one’s native language, words seem to be well separated from one another, like beads on a string. When listening to a foreign language, in contrast, words seem almost impossible to extract, as if there was only one bead on the same string. This contrast reveals that there are language-specific cues to segmentation. The puzzle, however, is that infants must be endowed with a language-independent mechanism for segmentation, as they ultimately solve the segmentation problem for any native language. Here, we approach the acquisition problem by asking whether there are language-independent cues to segmentation that might be available to even adult learners who have already acquired a native language. We show that adult learners recognize words in connected speech when only prosodic cues to word-boundaries are given from languages unfamiliar to the participants. In both artificial and natural speech, adult English speakers, with no prior exposure to the test languages, readily recognized words in natural languages with critically different prosodic patterns, including French, Turkish and Hungarian. We suggest that, even though languages differ in their sound structures, they carry universal prosodic characteristics. Further, these language-invariant prosodic cues provide a universally accessible mechanism for finding words in connected speech. These cues may enable infants to start acquiring words in any language even before they are fine-tuned to the sound structure of their native language.     

Newborns are sensitive to multiple cues for word segmentation in continuous speech

Before infants can learn words, they must identify those words in continuous speech. Yet, the speech signal lacks obvious boundary markers, which poses a potential problem for language acquisition (Swingley, Philos Trans R Soc Lond. Series B, Biol Sci 364 (1536), 3617–3632, 2009). By the middle of the first year, infants seem to have solved this problem (Bergelson & Swingley, Proc Natl Acad Sci 109 (9), 3253–3258, 2012; Jusczyk & Aslin, Cogn Psychol 29 , 1–23, 1995), but it is unknown if segmentation abilities are present from birth, or if they only emerge after sufficient language exposure and/or brain maturation. Here, in two independent experiments, we looked at two cues known to be crucial for the segmentation of human speech: the computation of statistical co‐occurrences between syllables and the use of the language's prosody. After a brief familiarization of about 3 min with continuous speech, using functional near‐infrared spectroscopy, neonates showed differential brain responses on a recognition test to words that violated either the statistical (Experiment 1) or prosodic (Experiment 2) boundaries of the familiarization, compared to words that conformed to those boundaries. Importantly, word recognition in Experiment 2 occurred even in the absence of prosodic information at test, meaning that newborns encoded the phonological content independently of its prosody. These data indicate that humans are born with operational language processing and memory capacities and can use at least two types of cues to segment otherwise continuous speech, a key first step in language acquisition.     

An interaction between prosody and statistics in the segmentation of fluent speech

Sensitivity to prosodic cues might be used to constrain lexical search. Indeed, the prosodic organization of speech is such that words are invariably aligned with phrasal prosodic edges, providing a cue to segmentation. In this paper we devise an experimental paradigm that allows us to investigate the interaction between statistical and prosodic cues to extract words from a speech stream. We provide evidence that statistics over the syllables are computed independently of prosody. However, we also show that trisyllabic sequences with high transition probabilities that straddle two prosodic constituents appear not to be recognized. Taken together, our findings suggest that prosody acts as a filter, suppressing possible word-like sequences that span prosodic constituents.     

On the lateralization of emotional prosody: an event-related functional MR investigation

In order to investigate the lateralization of emotional speech we recorded the brain responses to three emotional intonations in two conditions, i.e., "normal" speech and "prosodic" speech (i.e., speech with no linguistic meaning, but retaining the 'slow prosodic modulations' of speech). Participants listened to semantically neutral sentences spoken with a positive, neutral, or negative intonation in both conditions and judged how positive, negative, or neutral the intonation was on a five-point scale. Core peri-sylvian language areas, as well as some frontal and subcortical areas were activated bilaterally in the normal speech condition. In contrast, a bilateral fronto-opercular region was active when participants listened to prosodic speech. Positive and negative intonations elicited a bilateral fronto-temporal and subcortical pattern in the normal speech condition, and more frontal activation in the prosodic speech condition. The current results call into question an exclusive right hemisphere lateralization of emotional prosody and expand patient data on the functional role of the basal ganglia during the perception of emotional prosody.     

The semantics of prosody: acoustic and perceptual evidence of prosodic correlates to word meaning

This investigation examined whether speakers produce reliable prosodic correlates to meaning across semantic domains and whether listeners use these cues to derive word meaning from novel words. Speakers were asked to produce phrases in infant-directed speech in which novel words were used to convey one of two meanings from a set of antonym pairs (e.g., big/small). Acoustic analyses revealed that some acoustic features were correlated with overall valence of the meaning. However, each word meaning also displayed a unique acoustic signature, and semantically related meanings elicited similar acoustic profiles. In two perceptual tests, listeners either attempted to identify the novel words with a matching meaning dimension (picture pair) or with mismatched meaning dimensions. Listeners inferred the meaning of the novel words significantly more often when prosody matched the word meaning choices than when prosody mismatched. These findings suggest that speech contains reliable prosodic markers to word meaning and that listeners use these prosodic cues to differentiate meanings. That prosody is semantic suggests a reconceptualization of traditional distinctions between linguistic and nonlinguistic properties of spoken language.     

The neural underpinnings of prosody in autism

This study examines the processing of prosodic cues to linguistic structure and to affect, drawing on fMRI and behavioral data from 16 high-functioning adolescents with autism spectrum disorders (ASD) and 11 typically developing controls. Stimuli were carefully matched on pitch, intensity, and duration, while varying systematically in conditions of affective prosody (angry versus neutral speech) and grammatical prosody (questions versus statement). To avoid conscious attention to prosody, which normalizes responses in young people with ASD, the implicit comprehension task directed attention to semantic aspects of the stimuli. Results showed that when perceiving prosodic cues, both affective and grammatical, activation of neural regions was more generalized in ASD than in typical development, and areas recruited reflect heightened reliance on cognitive control, reading of intentions, attentional management, and visualization. This broader recruitment of executive and “mind-reading” brain areas for a relative simple language-processing task may be interpreted to suggest that speakers with high-functioning autism (HFA) have developed less automaticity in language processing and may also suggest that “mind-reading” or theory of mind deficits are intricately bound up in language processing. Data provide support for both a right-lateralized as well as a bilateral model of prosodic processing in typical individuals, depending upon the function of the prosodic information.     

The neural underpinnings of prosody in autism

This study examines the processing of prosodic cues to linguistic structure and to affect, drawing on fMRI and behavioral data from 16 high-functioning adolescents with autism spectrum disorders (ASD) and 11 typically developing controls. Stimuli were carefully matched on pitch, intensity, and duration, while varying systematically in conditions of affective prosody (angry versus neutral speech) and grammatical prosody (questions versus statement). To avoid conscious attention to prosody, which normalizes responses in young people with ASD, the implicit comprehension task directed attention to semantic aspects of the stimuli. Results showed that when perceiving prosodic cues, both affective and grammatical, activation of neural regions was more generalized in ASD than in typical development, and areas recruited reflect heightened reliance on cognitive control, reading of intentions, attentional management, and visualization. This broader recruitment of executive and "mind-reading" brain areas for a relative simple language-processing task may be interpreted to suggest that speakers with high-functioning autism (HFA) have developed less automaticity in language processing and may also suggest that "mind-reading" or theory of mind deficits are intricately bound up in language processing. Data provide support for both a right-lateralized as well as a bilateral model of prosodic processing in typical individuals, depending upon the function of the prosodic information.     

Understanding speaker attitudes from prosody by adults with Parkinson's disease

The ability to interpret vocal (prosodic) cues during social interactions can be disrupted by Parkinson's disease, with notable effects on how emotions are understood from speech. This study investigated whether PD patients who have emotional prosody deficits exhibit further difficulties decoding the attitude of a speaker from prosody. Vocally inflected but semantically nonsensical ‘pseudo‐utterances’ were presented to listener groups with and without PD in two separate rating tasks. Task 1 required participants to rate how confident a speaker sounded from their voice and Task 2 required listeners to rate how polite the speaker sounded for a comparable set of pseudo‐utterances. The results showed that PD patients were significantly less able than HC participants to use prosodic cues to differentiate intended levels of speaker confidence in speech, although the patients could accurately detect the polite/impolite attitude of the speaker from prosody in most cases. Our data suggest that many PD patients fail to use vocal cues to effectively infer a speaker's emotions as well as certain attitudes in speech such as confidence, consistent with the idea that the basal ganglia play a role in the meaningful processing of prosodic sequences in spoken language ( Pell & Leonard, 2003 ).     

Guess who? Children use prosody to infer intended listeners

This study examined the relative influence of prosody and semantic content in children's inferences about intended listeners. Children (n = 72), who ranged in age from 5 to 10 years, heard greetings with prosody and content that was either infant or adult directed and chose the intended listener from amongst an infant or an adult. While content affected all children's choices, the effect of prosody was stronger (at least, for children aged 7–10 years). For conditions in which prosodic cues were suggestive of one listener, and content cues, another, children aged 7–10 years chose the listener according to prosody. In contrast, the youngest age group (5‐ to 6‐year‐olds) chose listeners at chance levels in these incongruent conditions. While prosodic cues were most influential in determining children's choices, their ratings of how certain they felt about their choices indicated that content nonetheless influenced their thinking about the intended listener. Results are the first to show the unique influence of prosody in children's thinking about appropriate speech styles. Findings add to work showing children's ability to use prosody to make inferences about speakers' communicative intentions.     

Impaired processing of prosodic and musical patterns after right hemisphere damage

The distinction between the processing of musical information and segmental speech information (i.e., consonants and vowels) has been much explored. In contrast, the relationship between the processing of music and prosodic speech information (e.g., intonation) has been largely ignored. We report an assessment of prosodic perception for an amateur musician, KB, who became amusic following a right-hemisphere stroke. Relative to matched controls, KB's segmental speech perception was preserved. However, KB was unable to discriminate pitch or rhythm patterns in linguistic or musical stimuli. He was also impaired on prosodic perception tasks (e.g., discriminating statements from questions). Results are discussed in terms of common neural mechanisms that may underlie the processing of some aspects of both music and speech prosody.     

Suprasegmental information affects processing of talking faces at birth

From birth, newborns show a preference for faces talking a native language compared to silent faces. The present study addresses two questions that remained unanswered by previous research: (a) Does the familiarity with the language play a role in this process and (b) Are all the linguistic and paralinguistic cues necessary in this case? Experiment 1 extended newborns’ preference for native speakers to non-native ones. Given that fetuses and newborns are sensitive to the prosodic characteristics of speech, Experiments 2 and 3 presented faces talking native and nonnative languages with the speech stream being low-pass filtered. Results showed that newborns preferred looking at a person who talked to them even when only the prosodic cues were provided for both languages. Nonetheless, a familiarity preference for the previously talking face is observed in the “normal speech” condition (i.e., Experiment 1) and a novelty preference in the “filtered speech” condition (Experiments 2 and 3). This asymmetry reveals that newborns process these two types of stimuli differently and that they may already be sensitive to a mismatch between the articulatory movements of the face and the corresponding speech sounds.     

Untrained speakers' use of prosody in syntactic disambiguation and listeners' interpretations

We investigated how naively produced prosody affects listeners' end interpretations of ambiguous utterances. Non-professional speakers who were unaware of any ambiguity produced ambiguous sentences couched in short, unambiguous passages. In a forced-choice task, listeners could not tell which context the isolated ambiguous sentences came from (Exp. 1). However, listeners were able to correctly paraphrase the least ambiguous subset of these utterances, showing that prosody can be used to resolve ambiguity (Exp. 2). Nonetheless, in everyday language use, both prosody and context are available to interpret speech. When the least ambiguous sentences were cross-spliced into contexts biasing towards their original interpretations or into contexts biasing towards their alternative interpretations, answers to content questions about the ambiguous sentence, confidence ratings, and ratings of naturalness all indicated that prosody is ignored when context is available (Exp. 3). Although listeners can use prosody to interpret ambiguous sentences, they generally do not, and this makes sense in light of the frequent lack of reliable prosodic cues in everyday speech. Received: 3 April 1998 / Accepted: 21 October 1998     

韵律特征研究

介绍从知觉、认知和语料库分析角度对汉语韵律特征进行的一系列研究。（1）韵律特征知觉：用实验心理学和知觉标注的语料库分析方法,研究汉语语调和音高下倾与降阶问题,语句和语篇中知觉可以区分的韵律层级及相关的声学线索。研究结果支持汉语语调的双线模型理论和语句音高下倾的存在;证明语篇中知觉可以区分的韵律边界是小句、句子和段落,及其知觉相关的声学线索。（2）韵律特征与其他语言学结构的关系：在标注的语料库的基础上,用常规统计方法研究语句常规重音分布规律、语篇信息结构与重音的关系、并用决策树方法研究根据文本信息确定韵律短语边界和焦点的规则。（3）韵律特征在语篇理解中的作用：用实验心理学方法和脑电指标研究韵律对语篇信息整合和指代理解的影响,揭示其作用的认知和神经机制。讨论了这些研究结果对语音工程、语音学理论和心理语言学研究的实践和理论意义     

Attentional engagement during syllable discrimination: The role of salient prosodic cues in 6- to 8-month-old infants

Prosodic cues drive speech segmentation and guide syllable discrimination. However, less is known about the attentional mechanisms underlying an infant's ability to benefit from prosodic cues. This study investigated how 6- to 8-month-old Italian infants allocate their attention to strong vs. weak syllables after familiarization with four repeats of a single CV sequence with alternating strong and weak syllables (different syllables on each trial). In the discrimination test-phase, either the strong or the weak syllable was replaced by a pure tone matching the suprasegmental characteristics of the segmental syllable, i.e., duration, loudness and pitch, whereas the familiarized stimulus was presented as a control. By using an eye-tracker, attention deployment (fixation times) and cognitive resource allocation (pupil dilation) were measured under conditions of high and low saliency that corresponded to the strong and weak syllabic changes, respectively. Italian learning infants were found to look longer and also to show, through pupil dilation, more attention to changes in strong syllable replacement rather than weak syllable replacement, compared to the control condition. These data offer insights into the strategies used by infants to deploy their attention towards segmental units guided by salient prosodic cues, like the stress pattern of syllables, during speech segmentation.     

口语加工中的词语切分线索

词是语言的基本结构单位, 对词语进行切分是语言加工的重要步骤。口语语流中的切分线索来自于语音、语义和语法三个方面。语音线索包括概率信息、音位配列规则和韵律信息, 韵律信息中还包括词重音、时长和音高等内容, 这些线索的使用在接触语言的早期阶段就逐渐被个体所掌握, 而且在不同的语言背景下有一定的特异性。语法和语义线索属于较高级的线索机制, 主要作用于词语切分过程的后期。后续研究应从语言的毕生发展和语言的特异性两个方面考察口语语言加工中的词语切分线索。     

Developmental changes in sensitivity to vocal paralanguage

Developmental changes in children’s sensitivity to the role of acoustic variation in the speech stream in conveying speaker affect (vocal paralanguage) were examined. Four‐, 7‐ and 10‐year‐olds heard utterances in three formats: low‐pass filtered, reiterant, and normal speech. The availability of lexical and paralinguistic information varied across these three formats in a way that required children to base their judgments of speaker affect on different configurations of cues in each format. Across ages, the best performance was obtained when a rich array of acoustic cues was present and when there was no competing lexical information. Four‐year‐olds performed at chance when judgments had to be based solely on speech prosody in the filtered format and they were unable to selectively attend to paralanguage when discrepant lexical cues were present in normal speech. Seven‐year‐olds were significantly more sensitive to the paralinguistic role of speech prosody in filtered speech than were 4‐year‐olds and there was a trend toward greater attention to paralanguage when lexical and paralinguistic cues were inconsistent in normal speech. An integration of the ability to utilize prosodic cues to speaker affect with attention to paralanguage in cases of lexical/paralinguistic discrepancy was observed for 10‐year‐olds. The results are discussed in terms of the development of a perceptual bias emerging out of selective attention to language.     

Prosodic boundaries, comma rules, and brain responses: the closure positive shift in ERPs as a universal marker for prosodic phrasing in listeners and readers

Just as the false comma in this sentence, shows punctuation can influence sentence processing considerably. Pauses and other prosodic cues in spoken language serve the same function of structuring the sentence in smaller phrases. However, surprisingly little effort has been spent on the question as to whether both phenomena rest on the same mechanism and whether they are equally efficient in guiding parsing decisions. In a recent study, we showed that auditory speech boundaries evoke a specific positive shift in the listeners' event-related brain potentials (ERPs) that indicates the sentence segmentation and resulting changes in the understanding of the utterance (Steinhauer et al., 1999a). Here, we present three ERP reading experiments demonstrating that the human brain processes commas in a similar manner and that comma perception depends crucially on the reader's individual punctuation habits. Main results of the study are: (1) Commas can determine initial parsing as efficiently as speech boundaries because they trigger the same prosodic phrasing covertly, although phonological representations seem to be activated to a lesser extent. (2) Independent of the input modality, this phrasing is reflected online by the same ERP component, namely the Closure Positive Shift (CPS). (3) Both behavioral and ERP data suggest that comma processing varies with the readers' idiosyncratic punctuation habits. (4) A combined auditory and visual ERP experiment shows that the CPS is also elicited both by delexicalized prosody and while subjects replicate prosodic boundaries during silent reading. (5) A comma-induced reversed garden path turned out to be much more difficult than the classical garden path. Implications for psycholinguistic models and future ERP research are discussed.     

Lexical and sublexical units in speech perception

Saffran, Newport, and Aslin (1996a) found that human infants are sensitive to statistical regularities corresponding to lexical units when hearing an artificial spoken language. Two sorts of segmentation strategies have been proposed to account for this early word-segmentation ability: bracketing strategies, in which infants are assumed to insert boundaries into continuous speech, and clustering strategies, in which infants are assumed to group certain speech sequences together into units ( Swingley, 2005 ). In the present study, we test the predictions of two computational models instantiating each of these strategies i.e., Serial Recurrent Networks: Elman, 1990 ; and Parser: Perruchet & Vinter, 1998 in an experiment where we compare the lexical and sublexical recognition performance of adults after hearing 2 or 10 min of an artificial spoken language. The results are consistent with Parser's predictions and the clustering approach, showing that performance on words is better than performance on part-words only after 10 min. This result suggests that word segmentation abilities are not merely due to stronger associations between sublexical units but to the emergence of stronger lexical representations during the development of speech perception processes.     

Co‐occurrence statistics as a language‐dependent cue for speech segmentation

To what extent can language acquisition be explained in terms of different associative learning mechanisms? It has been hypothesized that distributional regularities in spoken languages are strong enough to elicit statistical learning about dependencies among speech units. Distributional regularities could be a useful cue for word learning even without rich language‐specific knowledge. However, it is not clear how strong and reliable the distributional cues are that humans might use to segment speech. We investigate cross‐linguistic viability of different statistical learning strategies by analyzing child‐directed speech corpora from nine languages and by modeling possible statistics‐based speech segmentations. We show that languages vary as to which statistical segmentation strategies are most successful. The variability of the results can be partially explained by systematic differences between languages, such as rhythmical differences. The results confirm previous findings that different statistical learning strategies are successful in different languages and suggest that infants may have to primarily rely on non‐statistical cues when they begin their process of speech segmentation.     

Infant directed speech and the development of speech perception: Enhancing development or an unintended consequence?

Infant directed speech (IDS) is a speech register characterized by simpler sentences, a slower rate, and more variable prosody. Recent work has implicated it in more subtle aspects of language development. Kuhl et al. (1997) demonstrated that segmental cues for vowels are affected by IDS in a way that may enhance development: the average locations of the extreme “point” vowels (/a/, /i/ and /u/) are further apart in acoustic space. If infants learn speech categories, in part, from the statistical distributions of such cues, these changes may specifically enhance speech category learning. We revisited this by asking (1) if these findings extend to a new cue (Voice Onset Time, a cue for voicing); (2) whether they extend to the interior vowels which are much harder to learn and/or discriminate; and (3) whether these changes may be an unintended phonetic consequence of factors like speaking rate or prosodic changes associated with IDS. Eighteen caregivers were recorded reading a picture book including minimal pairs for voicing (e.g., beach/peach) and a variety of vowels to either an adult or their infant. Acoustic measurements suggested that VOT was different in IDS, but not in a way that necessarily supports better development, and that these changes are almost entirely due to slower rate of speech of IDS. Measurements of the vowel suggested that in addition to changes in the mean, there was also an increase in variance, and statistical modeling suggests that this may counteract the benefit of any expansion of the vowel space. As a whole this suggests that changes in segmental cues associated with IDS may be an unintended by-product of the slower rate of speech and different prosodic structure, and do not necessarily derive from a motivation to enhance development.