The role of “chirp” identification in duplex perception

Duplex perception occurs when the phonetically distinguishing transitions of a syllable are presented to one ear and the rest of the syllable (the “base”) is simultaneously presented to the other ear. Subjects report hearing both a nonspeech “chirp” and a speech syllable correctly cued by the transitions. In two experiments, we compared phonetic identification of intact syllables, duplex percepts, isolated transitions, and bases. In both experiments, subjects were able to identify the phonetic information encoded into isolated transitions in the absence of an appropriate syllabic context. Also, there was no significant difference in phonetic identification of isolated transitions and duplex percepts. Finally, in the second experiment, the category boundaries from identification of isolated transitions and duplex percepts were not significantly different from each other. However, both boundaries were statistically different from the category boundary for intact syllables. Taken together, these results suggest that listeners do not need to perceptually integrate F2 transitions or F2 and F3 transition pairs with the base in duplex perception. Rather, it appears that listeners identify the chirps as speech without reference to the base.     

Some differences between phonetic and auditory modes of perception

When third-formant transitions are appropriately incorporated into an acoustic syllable, they provide critical support for the phonetic percepts we call [d] and [g], but when presented in isolation they are perceived as time-varying ‘chirps’. In the present experiment, both modes of perception were made available simultaneously by presenting the third-format transitions to one ear and the remainder of the acoustic syllable to the other. On the speech side of this duplex percept, where the transitions supported the perception of stop-vowel syllables, perception was categorical and influenced by the presence of a preposed [al] or [ar]. On the nonspeech side, where the same transitions were heard as ‘chirps’, perception was continuous and free of influence from the preposed syllables. As both differences occurred under conditions in which the acoustic input was constant, we should suppose that they reflect the different properties of auditory and phonetic modes of perception.     

The Phonetic Integration of Speech and Non-speech Sounds: Effects of Perceived Location

The third-formant (F3) transition of a three-formant /da/ or /ga/ syllable was extracted and replaced by sine-wave transitions that followed the F3 centre frequency. The syllable without the F3 transition (base) was always presented at the left ear, and a /da/ (falling) or /ga/ (rising) sine-wave transition could be presented at either the left, the right, or both ears. The listeners perceived the base as a syllable, and the sine-wave transition as a non-speech whistle, which was lateralized near the left ear, the right ear, or the middle of the head, respectively. In Experiment 1, the sine-wave transition strongly influenced the identity of the syllable only when it was lateralized at the same ear as the base (left ear). Phonetic integration between the base and the transitions became weak, but was not completely eliminated, when the latter was perceived near the middle of the head or at the opposite ear as the base (right ear). The second experiment replicated these findings by using duplex stimuli in which the level of the sine-wave transitions was such that the subjects could not reliably tell whether a /da/ or a /ga/ transition was present at the same ear as the base. This condition was introduced in order to control for the possibility that the subjects could have identified the syallables by associating a rising or falling transition presented at the left ear with a /da/ or /ga/ percept. Alternative suggestions about the relation between speech and non-speech perceptual processes are discussed on the basis of these results.     

Duplex perception of cues for stop consonants: Evidence for a phonetic mode

When the (vocalic) formant transitions appropriate for the stops in a synthetic approximation to [spa] or [sta] are presented to one ear and the remainder of the acoustic pattern to the other, listeners report a duplex percept. One side of the duplexity is the same coherent syllable ([spa] or [sta]) that is perceived when the pattern is presented in its original, undivided form; the other is a nonspeech chirp that corresponds to what the transitions sound like in isolation. This phenomenon is here used to determine why, in the case of stops, silence is an important cue. The results show that the silence cue affects the formant transitions differently when, on the one side of the duplex percept, the transitions support the perception of stop consonants, and when, on the other, they are perceived as nonspeech chirps. This indicates that the effectiveness of the silence cue is owing to distinctively phonetic (as against generally auditory) processes.     

ON THE IMMEDIACY OF PHONETIC PERCEPTION

Abstract— If, as we believe, language is a specialization alt the way down to its roots, then perception of its consonantal elements should be immediately phonetic, not as in the conventional view, a secondary translation from percepts of an auditory sort. Supporting observations come from an experiment in which formant transitions that distinguish [da] and [gal were presented as sinusoids and combined with a synthetic syllable made of resonances, thus causing the auditory system to treat these acoustically incoherent parts as different sources Evidence for the source difference was varied by changing the intensity of the sinusoids relative to the remainder of the syllable. Over the greater part of a 60-dB range, listeners accurately identified the consonants, indicating that they had integrated the stimuli according to a coherence that existed only in the phonetic domain. At the lowest intensities, indeed, the consonants were accurately identified, even though the whistles—the normal responses to the sinusoids—were not. There followed then a range over which perception was duplex. Both consonants and whistles were accurately identified At the highest intensities, phonetic integration failed, but accurate perception of the whistles was maintained. That the phonetic percept was present when as auditory counterpart was absent, and vice versa, is evidence that the phonetic percept is independent of its auditory counterpart and not a translation from it, as is the fact that the two percepts followed very different courses in response to the experimental variable.     

Limits on phonetic integration in duplex perception

The telling fact about duplex perception is that listeners integrate into a unitary phonetic percept signals that are coherent from a phonetic point of view, even though the signals are, on purely auditory grounds, separate sources. Here we explore the limits on the integration of a sinusoidal consonant cue (theF3 transition for [da] vs. [ga]) with the resonances of the remainder of the syllable. Perceiving duplexly, listeners hear the whistle of the sinusoid, but also the [da] and [ga] for which the sinusoid provides the critical information. In the first experiment, phonetic integration was significantly reduced, but not to zero, by a precursor that extended the transition cue forward in time so that it started 50 msec before the cue. The effect was the same above and below the duplexity threshold (the intensity of sinusoid in the combined pattern at which the whistle was just barely audible). In the second experiment, integration was reduced once again by the precursor, and also, but only below the duplexity threshold, by harmonics of the cues that were simultaneous with it. The third experiment showed that the simultaneous harmonics reduced phonetic integration only by serving as distractors while also permitting the conclusion that the precursor produced its effects by making the cue part of a coherent and competing auditory pattern, and so “capturing” it. The fourth experiment supported this interpretation by showing that for some subjects the amount of capture was reduced when the capturing tone was itself captured by being made part of a tonal complex. The results support the assumption that the independent phonetic system will integrate across disparate sources according to the cohesive power of that system as measured against the evidence for separate sources.     

Duplex perception: some initial findings concerning its neural basis

Duplex perception is the simultaneous perception of a speech syllable and of a nonspeech "chirp," and occurs when a single formant transition and the remainder (the "base") of a synthetic syllable are presented to different ears. The current study found a slight but nonsignificant advantage for correct labeling of the fused syllable when the chirp was presented to the left ear. This advantage was amplified in the performance of a "split-brain" subject. A subject with a left pontine lesion performed at chance level when the chirp was presented to her left ear. These findings suggest that some, if not complete, ipsilateral suppression does occur in the dichotic fusion procedure, and that identification of the fused syllable is maximal when the left hemisphere fully processes the linguistic characteristics of the base (through contralateral presentation), and at least minimally processes the frequency transition information of the chirp (through ipsilateral presentation).     

Auditory property detectors and processing place features in stop consonants

The effects of selective adaptation on the perception of consonant-vowel (CV) stimuli varying in place of production was studied under two conditions. In the first condition, repeated presentation of a CV syllable produced an adaptation effect resulting in a shift in the locus of the phonetic boundary between [ba] and [da]. This result replicated previously reported findings. However, in the second condition, an adaptation effect was obtained on this same test series when the critical acoustic information (i.e., formant transitions) was present in final position of a VC speech-like syllable. These latter results support an auditory account of selective adaptation based on the spectral similarity of the adapting stimuli and test series rather than a more abstract linguistic account based on phonetic identity.     

Use of visual information in speech perception: evidence for a visual rate effect both with and without a McGurk effect

The McGurk effect, where an incongruent visual syllable influences identification of an auditory syllable, does not always occur, suggesting that perceivers sometimes fail to use relevant visual phonetic information. We tested whether another visual phonetic effect, which involves the influence of visual speaking rate on perceived voicing (Green & Miller, 1985), would occur in instances when the McGurk effect does not. In Experiment 1, we established this visual rate effect using auditory and visual stimuli matching in place of articulation, finding a shift in the voicing boundary along an auditory voice-onset-time continuum with fast versus slow visual speech tokens. In Experiment 2, we used auditory and visual stimuli differing in place of articulation and found a shift in the voicing boundary due to visual rate when the McGurk effect occurred and, more critically, when it did not. The latter finding indicates that phonetically relevant visual information is used in speech perception even when the McGurk effect does not occur, suggesting that the incidence of the McGurk effect underestimates the extent of audio-visual integration.     

Audiovisual presentation demonstrates that selective adaptation in speech perception is purely auditory

MRC Institute of Hearing Research, University of Nottingham, Nottingham NG72RD, England Both auditory and phonetic processes have been implicated by previous results from selective adaptation experiments using speech stimuli. It has proved difficult to dissociate their individual contributions because the auditory and phonetic structure of conventional acoustical stimuli are mutually predictive. In the present experiment, the necessary dissociation was achieved by using an audiovisual adaptor consisting of an acoustical [b?] synchronized to a video recording of a talker uttering the syllable [g?]. This stimulus was generally identified as one of the dentals [d?] or [??]. It produced an adaptation effect, measured with an acoustical [b?-d?] test continuum, identical in size and direction to that produced by an acoustical [b?]—an adaptor sharing its acoustical structure—and opposite in direction to that produced by an acoustical [d?]—an adaptor sharing its perceived phonetic identity. Thus, the result strongly suggests that auditory rather than phonetic levels of processing are influenced in selective adaptation.     

Spatial inattention abolishes voice adaptation

Adaptation to male voices causes a subsequent voice to be perceived as more female, and vice versa. Similar contrastive aftereffects have been reported for phonetic perception, and in vision for face perception. However, while aftereffects in the perception of phonetic features of speech have been reported to persist even when adaptors were processed inattentively, face aftereffects were previously reported to be abolished by inattention to adaptors. Here we demonstrate that auditory aftereffects of adaptation to voice gender are eliminated when the male and female adaptor voices are spatially unattended. Participants simultaneously heard gender-specific male or female adaptor voices in one ear and gender-neutral (androgynous) adaptor voices in the contralateral ear. They selectively attended to the adaptor voices in a designated ear, by either classifying voice gender (Exp. 1) or spoken syllable (Exp. 2). Voice aftereffects were found only if the gender-specific voices were spatially attended, suggesting capacity limits in the processing of voice gender for the unattended ear. Remarkably, gender-specific adaptors in the attended ear elicited comparable aftereffects in test voices, regardless of prior attention to voice gender or phonetic content. Thus, within the attended ear, voice gender was processed even when it was irrelevant for the task at hand, suggesting automatic processing of gender along with linguistic information. Overall, voice gender adaptation requires spatial, but not dimensional, selective attention.     

Discrimination of intensity differences on format transitions in and out of syllable context

The Ss were presented pairs of stimuli, /bae/s, /ae/s, or isolated transitions from /bae/s, which differed in the initial 60 msec of the signals by 0, 7.5, or 9 dB. In the syllable context, the intensity differences were discriminated essentially at chance; in both the vowel and isolated transition conditions, the intensity differences were discriminated essentially perfectly. This outcome suggests that after the acoustic features of a stop-consonant/vowel syllable have been recorded into a phonetic representation, the acoustic information is relatively inaccessible for recall from auditory short-term memory.     

Auditory spectral integration in nontraditional speech cues in diotic and dichotic listening

Underlying auditory processes in speech perception were explored. Specifically of interest were the stages of auditory processing involved in the integration of dynamic information in nontraditional speech cues such as the virtual formant transitions. These signals utilize intensity ratio cues and changes in spectral center-of-gravity (instead of the actual formant frequency transitions) to produce perceived F3 glides. 6 men and 8 women (M age = 24.2 yr., SD = 2.1), recruited through posted materials from graduate students at The Ohio State University, participated in two experiments. The results for frequency-based formant transitions (Exp. 1) indicated that spectral cues to syllable identification are combined at more central levels of auditory processing. However, when the components of the virtual formant stimuli were divided between the ears in a dichotic listening task (Exp. 2), the results indicated that auditory spectral integration may occur above the auditory periphery but at stages more intermediate rather than central.     

Neurological evidence in support of a specialized phonetic processing module

Event-related potentials (ERPs) were utilized to study brain activity while subjects listened to speech and nonspeech stimuli. The effect of duplex perception was exploited, in which listeners perceive formant transitions that are isolated as nonspeech "chirps," but perceive formant transitions that are embedded in synthetic syllables as unique linguistic events with no chirp-like sounds heard at all (Mattingly et al., 1971). Brain ERPs were recorded while subjects listened to and silently identified plain speech-only tokens, duplex tokens, and tone glides (perceived as "chirps" by listeners). A highly controlled set of stimuli was developed that represented equivalent speech and nonspeech stimulus tokens such that the differences were limited to a single acoustic parameter: amplitude. The acoustic elements were matched in terms of number and frequency of components. Results indicated that the neural activity in response to the stimuli was different for different stimulus types. Duplex tokens had significantly longer latencies than the pure speech tokens. The data are consistent with the contention of separate modules for phonetic and auditory stimuli.     

Processing of place information in stop consonants

One of the basic questior, s that models of speech perception must answer concerns the conditions under which various cues will be extracted from a stimulus and the nature of the mechanisms which mediate this process. Two selective adaptation experiments were carried out to explore this question for the phonetic feature of place of articulation in both syllableinitial and syllable-final positions. In the first experiment, CV and VC stimuli were constructed with complete overlap in their second- and third-formant transitions. Despite this essentially complete overlap, no adaptation effects were found for a VC adaptor and a CV test series (or vice versa). In the second experiment, various vowel, vowel-like, and VC-like adaptors were used. The VC-like adaptors did have a significant effect on the CV category boundary, while the vowel and vowel-like stimuli did not. These results are interpreted within both one- and twolevel models of selective adaptation. These models are distinguished by whether selective adaptation is assumed to affect a single auditory level of processing or to affect both an auditory level and a later phonetic level. However, both models incorporate detectors at the auditory level which respond whenever particular formant transitions are present. These auditory detectors are not sensitive to the position of the consonant transition information within the syllable.     

Sing that tune: Infants’ perception of melody and lyrics and the facilitation of phonetic recognition in songs

To better understand how infants process complex auditory input, this study investigated whether 11-month-old infants perceive the pitch (melodic) or the phonetic (lyric) components within songs as more salient, and whether melody facilitates phonetic recognition. Using a preferential looking paradigm, uni-dimensional and multi-dimensional songs were tested; either the pitch or syllable order of the stimuli varied. As a group, infants detected a change in pitch order in a 4-note sequence when the syllables were redundant (experiment 1), but did not detect the identical pitch change with variegated syllables (experiment 2). Infants were better able to detect a change in syllable order in a sung sequence (experiment 2) than the identical syllable change in a spoken sequence (experiment 1). These results suggest that by 11 months, infants cannot “ignore” phonetic information in the context of perceptually salient pitch variation. Moreover, the increased phonetic recognition in song contexts mirrors findings that demonstrate advantages of infant-directed speech. Findings are discussed in terms of how stimulus complexity interacts with the perception of sung speech in infancy.     

A reexamination of duplex perception evoked by intensity differences

Duplex perception is a term used to describe the simultaneous evocation of phonetic and nonphonetic percepts, a phenomenon held by some to reveal the operation of a specialized module for phonetic perception. In a widely cited experiment by Whalen and Liberman (1987), duplex percepts were created by replacing the third formant transition in a synthetic syllable with a sinusoidal tone glide and manipulating the level of the glide relative to the rest of the syllable. We discuss this study, and report four experiments. The first two made a systematic estimate of the “duplexity threshold”—the tone glide level at which a chirp-like sound could be heard as well as the speech-like syllable. In the third and fourth experiments, we used the results of the first to repeat and extend the experimental procedures in the original Whalen and Liberman study. We discuss the differences in outcome, and conclude that our results may be explained without the need to invoke a specialized phonetic module.     

The role of visual information in the processing of place and manner features in speech perception

Visual information provided by a talker's mouth movements can influence the perception of certain speech features. Thus, the "McGurk effect" shows that when the syllable (bi) is presented audibly, in synchrony with the syllable (gi), as it is presented visually, a person perceives the talker as saying (di). Moreover, studies have shown that interactions occur between place and voicing features in phonetic perception, when information is presented audibly. In our first experiment, we asked whether feature interactions occur when place information is specificed by a combination of auditory and visual information. Members of an auditory continuum ranging from (ibi) to (ipi) were paired with a video display of a talker saying (igi). The auditory tokens were heard as ranging from (ibi) to (ipi), but the auditory-visual tokens were perceived as ranging from (idi) to (iti). The results demonstrated that the voicing boundary for the auditory-visual tokens was located at a significantly longer VOT value than the voicing boundary for the auditory continuum presented without the visual information. These results demonstrate that place-voice interactions are not limited to situations in which place information is specified audibly.(ABSTRACT TRUNCATED AT 250 WORDS)     

Discrimination of intensity differences on formant-like transitions

Dorman (1974) studied the discrimination of intensity differences on formant transitions in and out of syllable context. He interpreted his results as suggesting that the acoustic features of his stop-consonant/vowel syllable were recoded into a phonetic representation, then stored in an inaccessible form of auditory short-term memory. The Dorman results are replicated with analogous pure-tone and FM-glide conditions. The results of both studies are explained in terms of specified acoustic properties of the signals and thus provide no evidence for a special phonetic recoding.     

Some effects of later-occurring information on the perception of stop consonant and semivowel

In three experiments, we determined how perception of the syllable-initial distinction between the stop consonant [b] and the semivowel [w], when cued by duration of formant transitions, is affected by parts of the sound pattern that occur later in time. For the first experiment, we constructed four series of syllables, similar in that each had initial formant transitions ranging from one short enough for [ba] to one long enough for [wa], hut different in overall syllable duration. The consequence in perception was that, as syllable duration increased, the [b-w] boundary moved toward transitions of longer duration. Then, in the second experiment, we increased the duration of the sound by adding a second syllable, [da], (thus creating [bada-wada]), and observed that lengthening the second syllable also shifted the perceived [b-w] boundary in the first syllable toward transitions of longer duration; however, this effect was small by comparison with that produced when the first syllable was lengthened equivalently. In the third experiment, we found that altering the structure of the syllable had an effect that is not to be accounted for by the concomitant change in syllable duration: lengthening the syllable by adding syllable-final transitions appropriate for the stop consonant [d] (thus creating [bad-wad]) caused the perceived [b-w] boundary to shift toward transitions of shorter duration, an effect precisely opposite to that produced when the syllable was lengthened to the same extent by adding steady-state vowel. We suggest that, in all these cases, the later-occurring information specifies rate of articulation and that the effect on the earlier-occurring cue reflects an appropriate perceptual normalization.