Interactions in processing segmental and suprasegmental features of speech

A speeded classification task was used to determine the nature of the dependency relation that exists during the processing of a segmental distinction in vowel quality and a suprasegmental distinction in pitch (Experiment 1) or loudness (Experiment 2) in a consonantvowel syllable. In both experiments, evidence was found for a mutual, symmetrical dependency between the processes underlying the analysis of the segmental and suprasegmental information in the syllable. This type of interaction pattern contrasts with the pattern of unidirectional dependency previously found for the analysis of a consonantal distinction and a suprasegmental distinction, in which the analysis of the consonantal information was found to be dependent on the analysis of the suprasegmental information (e.g., Wood, 1974). Together with these earlier findings, the present results clearly indicate that the form of the interaction between processes responsible for segmental and suprasegmental analysis is a function of the type of segmental information being analyzed. Future research must determine whether the distinct interaction patterns found thus far for consonants and vowels are due to a difference in phonetic class per se, i.e., consonant vs. vowel, or to a difference in the nature of the acoustic information specifying the consonant and vowel.     

Stimulus variability and processing dependencies in speech perception

Processing dependencies in speech perception between voice and phoneme were investigated using the Garner (1974) speeded classification procedure. Variability in the voice of the talker and in the cues to word-initial consonants were manipulated. The results showed that the processing of a talker's voice and the perception of voicing are asymmetrically dependent. In addition, when stimulus variability was increased in each dimension, the amount of orthogonal interference obtained for each dimension became significantly larger. The processing asymmetry between voice and phoneme was interpreted in terms of a parallel-contingent relationship of talker normalization processes to auditory-to-phonetic coding processes. The processing of voice information appears to be qualitatively different from the encoding of segmental phonetic information, although they are not independent. Implications of these results for current theories of speech perception are discussed.     

Dimensional interactions in language processing: investigating directions and levels of crosstalk

In this article we evaluate current models of language processing by testing speeded classification of stimuli comprising one linguistic and one nonlinguistic dimension. Garner interference obtains if subjects are slower to classify attributes on one dimension when an irrelevant dimension is varied orthogonally than when the irrelevant dimension is held constant. With certain linguistic-nonlinguistic pairings (e.g., Experiment 1: the words high and low spoken either loudly or softly), significant Garner interference obtained when either dimension was classified; this indicated two-directional crosstalk. With other pairings (e.g., Experiment 3: spoken vowels and loudness), only the nonlinguistic dimension (e.g., loudness) displayed interference, suggesting unidirectional crosstalk downstream from a phonemic/graphemic level of analysis. Collectively, these results indicate the interaction can occur either within or across levels of information processing, being directed toward either more advanced or more primitive processes. Although poorly explained by all current models of language processing, our results are strikingly inconsistent with models that posit autonomy among levels of processing.     

Processing two dimensions of nonspeech stimuli: the auditory-phonetic distinction reconsidered.

Nonspeech stimuli were varied along two dimensions--intensity and rise time. In a series of speeded classification tasks, subjected were asked to identify the stimuli in terms of one of these dimensions. Identification time for the dimension of rise time increased when there was irrelevant variation in intensity; however, identification of intensity was unaffected by irrelevant variation in rise time. When the two dimensions varied redundantly, identification time decreased. This pattern of results is virtually identical to that obtained previously for stimuli that vary along a linguistic and a nonlinguistic dimension. The present data, taken together with those from other studies using the same stimuli, suggest that the mechanisms underlying the auditory-phonetic distinction should be reconsidered. The results are also discussed in terms of general models of multidimensional information processing.     

Processing interactions between phonology and melody: Vowels sing but consonants speak

The aim of this study was to determine if two dimensions of song, the phonological part of lyrics and the melodic part of tunes, are processed in an independent or integrated way. In a series of five experiments, musically untrained participants classified bi-syllabic nonwords sung on two-tone melodic intervals. Their response had to be based on pitch contour, on nonword identity, or on the combination of pitch and nonword. When participants had to ignore irrelevant variations of the non-attended dimension, patterns of interference and facilitation allowed us to specify the processing interactions between dimensions. Results showed that consonants are processed more independently from melodic information than vowels are (Experiments 1-4). This difference between consonants and vowels was neither related to the sonority of the phoneme (Experiment 3), nor to the acoustical correlates between vowel quality and pitch height (Experiment 5). The implication of these results for our understanding of the functional relationships between musical and linguistic systems is discussed in light of the different evolutionary origins and linguistic functions of consonants and vowels.     

Perception of dynamic information for vowels in syllable onsets and offsets.

It has been demonstrated using the "silent-center" (SC) syllable paradigm that there is sufficient information in syllable onsets and offsets, taken together, to support accurate identification of vowels spoken in both citation-form syllables and syllables spoken in sentence context. Using edited natural speech stimuli, the present study examined the identification of American English vowels when increasing amounts of syllable onsets alone or syllable offsets alone were presented in their original sentence context. The stimuli were /d/-vowel-/d/ syllables spoken in a short carrier sentence by a male speaker. Listeners attempted to identify the vowels in experimental conditions that differed in the number of pitch periods presented and whether the pitch periods were from syllable onsets or syllable offsets. In general, syllable onsets were more informative than syllable offsets, although neither onsets nor offsets alone specified vowel identity as well as onsets and offsets together (SC syllables). Vowels differed widely in ease of identification; the diphthongized long vowels /e/, /ae/, /o/ were especially difficult to identify from syllable offsets. Identification of vowels as "front" or "back" was accurate, even from short samples of the syllable; however, vowel "height" was quite difficult to determine, again, especially from syllable offsets. The results emphasize the perceptual importance of time-varying acoustic parameters, which are the direct consequence of the articulatory dynamics involved in producing syllables.     

Processes underlying dimensional interactions: Correspondences between linguistic and nonlinguistic dimensions

In six experiments, we examined speeded classification when one dimension was linguistic and the other was nonlinguistic. In five of these, attributes on the dimensions corresponded meaningfully, having in common the concepts "high" and "low." For example, in Experiment 1, the visually presented words HI and LO were paired with high- or low-pitched tones; in Experiment 2, the dimensions were visual words and vertical position, in Experiment 3, they were spoken words and position, and in Experiments 4 and 5, spoken words and pitch. For each dimension in each pair, subjects suffered Garner interference when dimensions were varied orthogonally. Garner interference remained constant across 15 blocks of trials (Experiment 5). Subjects also showed significant congruity effects in all experiments, with attributes from congruent stimuli (e.g., HI/high pitch) classified faster than attributes from incongruent stimuli (e.g., HI/low pitch). These results differ from those obtained previously with noncorresponding pairs of linguistic-nonlinguistic dimensions. The results also differ from those obtained with traditional Stroop dimensions (colors and color words; Experiment 6), which showed minimal Garner interference and diminishing congruity effects across blocks of trials. We conclude that the interactions found here represent cross-talk between channels within a semantic level of processing. We contrast our view with current models of dimensional interaction.     

Interaction among auditory dimensions: timbre, pitch, and loudness

In two experiments, we examined whether or not pairs of auditory dimensions--timbre-loudness (Experiment 1) and timbre-pitch (Experiment 2)--interact in speeded classification. Subjects classified values from one dimension while the other dimension was (1) held constant (baseline), (2) varied orthogonally (filtering), or (3) correlated linearly. The subjects showed substantial Garner interference when classifying all dimensions--that is, poor performance at filtering relative to baseline. Timbre and loudness displayed redundancy gain (i.e., performance faster than baseline) when correlated positively, but redundancy loss (i.e., interference) when correlated negatively. Timbre and pitch displayed redundancy gain however dimensions were correlated. Both pairs of dimensions showed substantial effects of congruity: Attributes from one dimension were classified faster when paired with "congruent" attributes from the other dimension. The results are interpreted in terms of an interactive multichannel model of auditory processing.     

Investigation of the process underpinning vowel‐size correspondence

Vowel‐size correspondence is frequently reported in the literature: Namely, the vowels a and i tend to elicit bigger and smaller images, respectively. Previous studies have speculated that two factors may contribute to this vowel‐size correspondence: the acoustical features of vowels and the speaker's kinesthetic experience of producing them. However, these two factors have been investigated without being considered separately in previous research. In this study, we investigated the process underpinning vowel‐size correspondence by using speeded classification tasks and manipulating the two factors mentioned above separately in two experiments. The results of Experiment 1 indicate that a and i elicited bigger and smaller images even in the absence of kinesthetic experience. The results of Experiment 2 indicate that the proprioception of the size of the oral cavity on its own may not contribute to vowel‐size correspondence. Thus, the acoustic features of vowels mainly contribute to vowel‐size correspondence, although other possibilities are also discussed.     

Perception of dynamic information for vowels in syllable onsets and offsets

It has been demonstrated using the “silent-center” (SC) syllable paradigm that there is sufficient information in syllable onsets and offsets,taken together, to support accurate identification of vowels spoken in both citation-form syllables and syllables spoken in sentence context. Using edited natural speech stimuli, the present study examined the identification of American English vowels when increasing amounts of syllable onsetsalone or syllable offsetsalone were presented in their original sentence context. The stimuli were /d/-vowel-/d/ syllables spoken in a short carrier sentence by a male speaker. Listeners attempted to identify the vowels in experimental conditions that differed in the number of pitch periods presented and whether the pitch periods were from syllable onsets or syllable off-sets. In general, syllable onsets were more informative than syllable offsets, although neither onsets nor offsets alone specified vowel identity as well as onsets and offsets together (SC syllables). Vowels differed widely in ease of identification; the diphthongized long vowels /e/, /ae/, /o/ were especially difficult to identify from syllable offsets. Identification of vowels as “front” or “back” was accurate, even from short samples of the syllable; however, vowel "height" was quite difficult to determine, again, especially from syllable offsets. The results emphasize the perceptual importance of time-varying acoustic parameters, which are the direct consequence of the articulatory dynamics involved in producing syllables.     

Spectral discontinuities and the vowel length effect

Perception of voicing for stop consonants in consonant-vowel syllables can be affected by the duration of the following vowel so that longer vowels lead to more “voiced” responses. On the basis of several experiments, Green, Stevens, and Kuhl (1994) concluded that continuity of fundamental frequency (f0), but not continuity of formant structure, determined the effective length of the following vowel. In an extension of those efforts, we found here that both effects were critically dependent on particular f 0s and formant values. First, discontinuity inf0 does not necessarily preclude the vowel length effect because the effect maintains whenf0 changes from 200 to 100 Hz, and 200-Hz partials extend continuously through test syllables. Second, spectral discontinuity does preclude the vowel length effect when formant changes result in a spectral peak shifting to another harmonic. The results indicate that the effectiveness of stimulus changes for sustaining or diminishing the vowel length effect depends critically on particulars of spectral composition.     

The relation between discriminability and memory for vowels,consonants, and silent-center vowels

People remember lists of vowel-contrasting syllables better than lists that vary only in stop consonant identity. Most views suggest that this difference is due to the structure of immediate memory and the greater discriminability of vowels compared with consonants. In all of these views, there is a presumed systematic relationship between discriminability and recall so that the more discriminable an item, the better that item should be recalled. The 11 experiments reported here measured the relative discriminability of and compared serial recall for (1) intact syllables that varied only in the medial vowel, (2) intact syllables that varied only in the initial consonant, and (3) syllables with the center vowel replaced by silence (so-called silent-center vowels). When item discriminability, as measured by identification, was equated for consonant-contrasting and silent-center lists, serial recall performance was also equal. However, even when the vowels were less discriminable than the consonants or silentcenter vowels, serial recall performance for the vowels was still better. These results are problematic for theories based on acoustic discriminability but can be explained parsimoniously by Nairne’s (1990) feature model.     

Dimensional interactions in vibrotactile information processing

The effect of irrelevant dimensional variation on the processing of vibrotactile stimuli was measured. Six observers performed a speeded classification task with stimuli varying along the dimensions of pitch and loudness. Choice reaction times were obtained for stimuli differing on one dimension alone, on two correlated dimensions, or on two orthogonally varied dimensions. Compared to one-dimension performance, reaction times were faster in the correlated condition and slower in the orthogonal condition. In general, these findings agreed with similar experiments in other modalities, with the exception that the effects in this study tended to be stronger for cases in which loudness, rather than pitch, was the relevant dimension. The results are explained in terms of the integrality of pitch and loudness and of the relative discriminability of dimensions.     

The role of binaural and fundamental frequency difference cues in the identification of concurrently presented vowels

The relative importance of voice pitch and interaural difference cues in facilitating the recognition of both of two concurrently presented synthetic vowels was measured. The interaural difference cues used were an interaural time difference (400 μsec ITD), two magnitudes of interaural level difference (15 dB and infinite ILD), and a combination of ITD and ILD (400 μsec plus 15 dB). The results are analysed separately for those cases where both vowels are identical and those where they are different. When the two vowels are different, a voice pitch difference of one semitone is found to improve the percentage of correct reports of both vowels by 35.8% on average. However, the use of interaural difference cues results in an improvement of 11.5% on average when there is a voice pitch difference of one semitone, but only a non-significant 0.1% when there is no voice pitch difference. When the two vowels are identical, imposition of either a voice pitch difference or binaural difference reduces performance, in a subtractive manner. It is argued that the smaller size of the interaural difference effect is not due to a “ceiling effect” but is characteristic of the relative importance of the two kinds of cues in this type of experiment. The possibility that the improvement due to interaural difference cues may in fact be due to monaural processing is discussed. A control experiment is reported for the ITD condition, which suggests binaural processing does occur for this condition. However, it is not certain whether the improvement in the ILD condition is due to binaural processing or use of the improvement in signal-to-noise ratio for a single vowel at each ear.     

Dimensional interaction between color and pitch

In three experiments I investigated the nature of cross-modal dimensional interaction by testing speeded classification of the synesthetically corresponding dimensions of color (white-black) and pitch (high-low). Experiment 1 showed significant Garner interference when these dimensions were varied orthogonally--redundancy gain for positively correlated dimensions and redundancy loss for negatively correlated dimensions. Attributes from synesthetically congruent stimuli were classified faster than attributes from incongruent stimuli (a congruity effect). Experiment 2 tested a perceptual explanation of this interaction (i.e., that color and pitch are configural dimensions) by using Pomerantz's (1986) diagnostic (comparison of selective and divided attention performance). The configurality hypothesis received little support. Experiment 3 examined the effect of optional processes on color and pitch classification. The results suggest that partly strategic and partly mandatory components may constitute overall performance. Three alternative explanations of the color-pitch interaction--perceptual, semantic, and response based--are evaluated in the context of the present results.     

Processing interactions between segmental and suprasegmental information in native speakers of English and Mandarin Chinese

The processing interactions between segmental and suprasegmental information in native speakers of English and Mandarin Chinese were investigated in a speeded classification task. Since in Chinese, unlike in English, tones convey lexically meaningful information, native speakers of these languages may process combinations of segmental and suprasegment,al information differently. Subjects heard consonant-vowel syllables varying on a consonantal (segmental) dimension and either a Mandarin Chinese or constant-pitch (non-Mandarin) suprasegmental dimension. The English listeners showed mutual integrality with the Mandarin Chinese stimuli, but not the constant-pitch stimuli. The native Chinese listeners processed these dimensions with mutual integrality for both the Mandarin Chinese and the constant-pitch stimuli. These results were interpreted in terms of the linguistic function and the structure of suprasegmental information in Chinese and English. The results suggest that the way listeners perceive speech depends on the interaction between the structure of the signal and the processing strategies of the listener.     

Identification of vowels in “vowelless” syllables

Traditionally, it has been held that the primary information for vowel identification is provided by formant frequencies in the quasi-steady-state portion of the spoken syllable. Recent search has advanced an alternative view that emphasizes the role of temporal factors and dynamic (time-varying) spectral information in determining the perception of vowels. Nine vowels spoken in /b/ + vowel + /b/ syllables were recorded. The syllables were modified electronically in several ways to suppress various sources of spectral and durational information. Two vowel-perception experiments were performed, testing subjects’ ability to identify vowels in these modified syllables. Results of both experiments revealed the importance of dynamic spectral information at syllable onset and offset (in its proper temporal relation) in permitting vowel identification. On the other hand, steady-state spectral information, deprived of its durational variation, was a poor basis for identification. Results constitute a challenge to traditional accounts of vowel perception and point toward important sources of dynamic information.     

Contextual effects in vowel perception II: Evidence for two processing mechanisms

Recent experiments have indicated that contrast effects can be obtained with vowels by anchoring a test series with one of the endpoint vowels. These contextual effects cannot be attributed to feature detector fatigue or to the induction of an overt response bias. In the present studies, anchored ABX discrimination functions and signal detection analyses of identification data (before and after anchoring) for an [i]-[I] vowel series were used to demonstrate that [i] and [I] anchoring produce contrast effects by affecting different perceptual mechanisms. The effects of [i] anchoring were to increase within-[i] category sensitivity, while [I] anchoring shifted criterion placements. When vowels were placed in CVC syllables to reduce available auditory memory, there was a significant decrease in the size of the [I]-anchor contrast effects. The magnitude of the [i]-anchor effect was unaffected by the reduction in vowel information available in auditory memory. These results suggest that [i] and [I] anchors affect mechanisms at different levels of processing. The [i] anchoring results may reflect normalization processes in speech perception that operate at an early level of perceptual processing, while the [I] anchoring results represent changes in response criterion mediated by auditory memory for vowel information.     

The structure of graphemic representations

The analysis of the spelling performance of a brain-damaged dysgraphic subject is reported. The subject's spelling performance was affected by various graphotactic factors, such as the distinction between consonant and vowel and graphosyllabic structure. For example, while the subject produced many consonant and vowel deletion errors when these were part of consonant and vowel clusters, respectively (e.g., sfondo----sondo; giunta----gunta), deletions were virtually never produced for single consonants flanked by two vowels (e.g., onesto----oesto) or for single vowels flanked by two consonants (e.g., tirare----trare). The demonstration that graphosyllabic factors affect spelling performance disconfirms the hypothesis that graphemic representations consist simply of linearly ordered sets of graphemes. It is concluded that graphemic representations are multidimensional structures: one dimension specifies the grapheme identities that comprise the spelling of a word; a second dimension specifies the consonant/vowel status of the graphemes; a third dimension represents the graphosyllabic structure of the grapheme string; and, a fourth dimension provides information about geminate features.     

Intrinsicf0 differences in spoken and sung vowels and their perception by listeners

We explore how listeners perceive distinct pieces of phonetic information that are conveyed in parallel by the fundamental frequency (f0) contour of spoken and sung vowels. In a first experiment, we measured differences inf0 of /i/ and /a/ vowels spoken and sung by unselected undergraduate participants. Differences in “intrinsicf0” (withf0 of /i/ higher than of /a/) were present in spoken and sung vowels; however, differences in sung vowels were smaller than those in spoken vowels. Four experiments tested a hypothesis that listeners would not hear the intrinsicf0 differences as differences in pitch on the vowel, because they provide information, instead, for production of a closed or open vowel. The experiments provide clear evidence of “parsing” of intrinsicf0 from thef0 that contributes to perceived vowel pitch. However, only some conditions led to an estimate of the magnitude of parsing that closely matched the magnitude of produced intrinsicf0 differences.