Comparison of the effect of onset asynchrony on auditory grouping in pitch matching and vowel identification

Previous experiments have shown that when a slightly mistuned harmonic of a complex tone starts more than about 80 msec before the remaining components, it makes a reduced contribution to the pitch of the complex. This contribution decreases to zero by about 300-msec onset asynchrony. In vowel perception, however, analogous experiments have shown that a much shorter asynchrony (around 40 msec) is enough to ensure that a component does not influence a vowel’s phonemic category. The three experiments reported here demonstrate that this difference in the utility of onset time as a grouping cue does not arise because of differences in stimulus structure, but rather is due to the perceptual task. They show that the onset asynchrony needed in a pitch-matching experiment to remove the contribution that a mistuned component makes to the pitch of a vowel is the same as that needed to remove the contribution to the pitch of a flat-spectrum complex tone. They further show that a much smaller onset asynchrony is needed to perceptually remove the same harmonic from a vowel for the calculation of vowel quality. The implication of this result for models of auditory grouping is discussed.     

Inhibitory influences on asynchrony as a cue for auditory segregation

A harmonic that begins before the other harmonics contributes less than they do to vowel quality. This reduction can be partly reversed by accompanying the leading portion with a captor tone. This effect is usually interpreted as reflecting perceptual grouping of the captor with the leading portion. Instead, it has recently been proposed that the captor effect depends on broadband inhibition within the central auditory system. A test of psychophysical predictions based on this proposal showed that captor efficacy is (a) maintained for noise-band captors, (b) absent when a captor accompanies a harmonic that continues after the vowel, and (c) maintained for 80 ms or more over a gap between captor offset and vowel onset. These findings support and refine the inhibitory account.     

Grouping frequency components of vowels: When is a harmonic not a harmonic?

When one harmonic of a vowel starts before and stops after the others, its contribution to the vowel's phonetic quality is reduced. Two experiments demonstrate that this reduction cannot be attributed entirely to adaptation. The first experiment shows that a harmonic that starts at the same time as a short vowel but continues after the vowel has ended contributes almost as little to the vowel's phonetic quality as a harmonic that starts before but stops at the same time as the vowel. The second experiment shows that the small contribution to vowel quality of a harmonic that starts before a vowel can be increased by adding an additional tone that will in turn form a perceptual group with that part of the harmonic preceding the vowel. The experiments demonstrate that some perceptual grouping operations are performed before the first formant of a vowel is estimated from the amplitudes of its component harmonics.     

Auditory objects of attention: the role of interaural time differences.

The role of interaural time difference (ITD) in perceptual grouping and selective attention was explored in 3 experiments. Experiment 1 showed that listeners can use small differences in ITD between 2 sentences to say which of 2 short, constant target words was part of the attended sentence, in the absence of talker or fundamental frequency differences. Experiments 2 and 3 showed that listeners do not explicitly track components that share a common ITD. Their inability to segregate a harmonic from a target vowel by a difference in ITD was not substantially changed by the vowel being placed in a sentence context, where the sentence shared the same ITD as the rest of the vowel. The results indicate that in following a particular auditory sound source over time, listeners attend to perceived auditory objects at particular azimuthal positions rather than attend explicitly to those frequency components that share a common ITD.     

Vowel quality changes produced by surrounding tone sequences

In three experiments, we examined whether energy at the same frequency as one of a vowel's harmonics in the F1 region can be captured by a preceding or following sequence of tones. The position of the /I/-/E/ phoneme boundary along an F1 continuum was used to assess the extent of capture. The first two experiments showed that a sequence of tones at 500 Hz (56-msec duration at 10/sec) can perceptually remove added energy at 500 Hz from a steady vowel (F0 = 125 Hz) that forms part of the sequence. The effect is detectable with one preceding tone, asymptotes with four, and is greater when two tones follow the vowel than when none do. Rising and falling sequences of tones (at 62.5-Hz intervals or at whole-tone intervals) differ in their effect. Falling sequences behave much like constant tones at 500 Hz but with less effect, whereas rising sequences show no evidence of removing the added tone. The second experiment replicated the first and also showed that when the vowel is embedded in a rising or a falling sequence of tones that continue after it, the following tones have no effect. The third experiment suggested that the different effects found with rising versus falling sequences are qualitatively predictable on the basis of the additive effects of their constituent tones rather than by virtue of their contour. The experiments indicated that sequences of repeating tones are much more effective at capturing a harmonic from a vowel than are sequences that follow a simple pattern. This result may reflect the operation of a principle of least commitment in auditory grouping.     

Enhancement,extension, and reversal of the frequency selectivity effect

The influence of a frequency cue on judgments of whether or not a subsequent target incorporated a brief silent gap was examined. In Experiment 1, there was no predictive frequency relation and evidence of auditory inhibition of return was obtained with frequency repetitions, producing a facilitative effect at 175-msec stimulus onset asynchrony (SOA) and an inhibitory effect at 775-msec SOA. Relative to this baseline performance pattern, increasing the probability of a frequency match to .75 (Experiment 2) served to generate a beneficial effect of frequency repetitions at lengthy SOAs and to enlarge its magnitude at 175-msec SOA. In contrast, a reduction in the probability of a frequency match to .25 (Experiment 3) resulted in the elimination of any facilitative effect of repetition at 175-msec SOA and the development of an inhibitory effect at 475- and 1,075-msec SOA. These results establish that a frequency cue may engage both exogenous and endogenous attentional processes within 175 msec following its presentation.     

Disruption of visual short-term memory by changing-state auditory stimuli: The role of segmentation

Typically, serial recall performance can be disrupted by the presence of an irrelevant stream of background auditory stimulation, but only if the background stream changes over time (the auditory changing-state effect). It was hypothesized that segmentation of the auditory stream is necessary for changing state to be signified. In Experiment 1, continuous random pitch glides failed to disrupt serial recall, but glides interrupted regularly by silence brought about the usual auditory changing-state effect. In Experiment 2, a physically continuous stream of synthesized vowel sounds was found to have disruptive effects. In Experiment 3, the technique of auditory induction showed that preattentive organization rather than critical features of the sound could account for the disruption by glides. With pitch glides, silence plays a preeminent role in the temporal segmentation of the sound stream, but speech contains corr-elated-time-varying changes in frequency and amplitude that make silent intervals superfluous.     

Auditory streaming and the continuity illusion

In the present experiment, auditory stream organization was investigated in the presence of perceptually restored continuity. It was found that auditory streaming processes tend to yield the same perceptual organization independently of the presence or absence of perceptual restoration. Other observations include the dominance of frequency proximity over trajectory as a perceptual organization principle, and the effect of harmonic enrichment on perceptual grouping.     

声调在汉语音节感知中的作用

探讨超音段(如声调)与音段信息的共同作用机制, 对口语词汇识别研究具有重要的理论意义。有研究探讨了声调在口语词汇语义通达阶段的作用, 但在相对早期的音节感知阶段, 声调与声母、韵母的共同作用机制还缺乏系统的认识。本研究采用oddball实验范式, 通过两个行为实验考察了声调在汉语音节感知中的作用。实验1发现检测声调和声母变化的时间没有差异, 均比检测韵母变化的时间长, 表明在汉语音节感知中对声调的敏感性不及韵母。实验2发现声母和韵母的组合并没有明显优于对韵母的觉察, 但声调与声母或韵母的同时变化都促进了被试对偏差刺激的觉察, 表明声调通过与声母、韵母的结合来共同影响汉语音节的感知加工。研究结果在认知行为层面为声调在音节感知中的作用机制提供了直接的实验证据, 为进一步探讨超音段与音段信息共同作用的认知神经机制提供了基础。     

Sound enhances visual perception: cross-modal effects of auditory organization on vision

Six experiments demonstrated cross-modal influences from the auditory modality on the visual modality at an early level of perceptual organization. Participants had to detect a visual target in a rapidly changing sequence of visual distractors. A high tone embedded in a sequence of low tones improved detection of a synchronously presented visual target (Experiment 1), but the effect disappeared when the high tone was presented before the target (Experiment 2). Rhythmically based or order-based anticipation was unlikely to account for the effect because the improvement was unaffected by whether there was jitter (Experiment 3) or a random number of distractors between successive targets (Experiment 4). The facilitatory effect was greatly reduced when the tone was less abrupt and part of a melody (Experiments 5 and 6). These results show that perceptual organization in the auditory modality can have an effect on perceptibility in the visual modality.     

Auditory enhancement and the perception of concurrent vowels

Listeners identified both constituents of double vowels created by summing the waveforms of pairs of synthetic vowels with the same duration and fundamental frequency. Accuracy of identification was significantly above chance. Effects of introducing such double vowels by visual or acoustical precursor stimuli were examined. Precursors specified the identity of one of the two constituent vowels. Performance was scored as the accuracy with which the other vowel was identified. Visual precursors were standard English spellings of one member of the vowel pair; acoustical precursors were 1-sec segments of one member of the vowel pair. Neither visual precursors nor contralateral acoustical precursors improved performance over the condition with no precursor. Thus, knowledge of the identity of one of the constituents of a double vowel does not help listeners to identify the other constituent. A significant improvement in performance did occur with ipsilateral acoustical precursors, consistent with earlier demonstrations that frequency components which undergo changes in spectral amplitude achieve enhanced auditory prominence relative to unchanging components. This outcome demonstrates the joint but independent operation of auditory and perceptual processes underlying the ability of listeners to understand speech despite adversely peaked frequency responses in communication channels.     

Effects of visual attentional load on low-level auditory scene analysis

The sharing of processing resources between the senses was investigated by examining the effects of visual task load on auditory event-related brain potentials (ERPs). In Experiment 1, participants completed both a zero-back and a one-back visual task while a tone pattern or a harmonic series was presented. N1 and P2 waves were modulated by visual task difficulty, but neither mismatch negativity (MMN) elicited by deviant stimuli from the tone pattern nor object-related negativity (ORN) elicited by mistuning from the harmonic series was affected. In Experiment 2, participants responded to identity (what) or location (where) in vision, while ignoring sounds alternating in either pitch (what) or location (where). Auditory ERP modulations were consistent with task difficulty, rather than with task specificity. In Experiment 3, we investigated auditory ERP generation under conditions of no visual task. The results are discussed with respect to a distinction between process-general (N1 and P2) and processspecific (MMN and ORN) auditory ERPs.     

The perception of back vowels: Centre of gravity hypothesis

According to the formant centre of gravity (FCOG) hypothesis, two vowel formants in close proximity are merged during perceptual analysis, and their contribution to vowel quality depends on the centre of gravity of the formant cluster. Findings consistent with this hypothesis are that two formants can be replaced by a single formant of intermediate centre frequency, provided their separation is less than 3-3.5 Bark; and that changes in their relative amplitudes produce systematic shifts in vowel quality. In Experiment 1, listeners adjusted the frequencies of F1 and F2 in a synthesized 6-formant vowel (with the F1-F2 separation fixed at 250 Hz, i.e. less than 3 Bark) to find the best phonetic match to a reference vowel with modified formant amplitudes. Contrary to FCOG predictions, F2 attenuation did not produce lower frequency matches. Raising the amplitude of F2 led to predicted upward shifts in formant frequencies of the matched vowel, but with increased variability of matches for some stimuli. In Experiment 2, listeners identified synthesized vowels with a range of separations of F1 and F2. Formant amplitude manipulations had no effect on listeners' judgements when the fundamental frequency was low (125 Hz). Small shifts in vowel quality appeared for stimuli with a high fundamental (250 Hz), but the shifts were significantly larger for F1-F2 separations greater than 3.5 Bark. These effects of formant amplitude are qualitatively different from those observed with single-formant vowels and are generally incompatible with a formant-averaging mechanism.     

The effect of exogenous spatial attention on auditory information processing

This study investigated the effect of exogenous spatial attention on auditory information processing. In Experiments 1, 2 and 3, temporal order judgment tasks were performed to examine the effect. In Experiment 1 and 2, a cue tone was presented to either the left or right ear, followed by sequential presentation of two target tones. The subjects judged the order of presentation of the target tones. The results showed that subjects heard both tones simultaneously when the target tone, which was presented on the same side as the cue tone, was presented after the target tone on the opposite side. This indicates that spatial exogenous attention was aroused by the cue tone, and facilitated subsequent auditory information processing. Experiment 3 examined whether both cue position and frequency influence the resulting information processing. The same effect of spatial attention was observed, but the effect of attention to a certain frequency was only partially observed. In Experiment 4, a tone fusion judgment task was performed to examine whether the effect of spatial attention occurred in the initial stages of hearing. The result suggests that the effect occurred in the later stages of hearing.     

Verbal short-term memory as an articulatory system: Evidence from an alternative paradigm

In a series of seven experiments, the role of articulatory rehearsal in verbal short-term memory was examined via a shadowing-plus-recall paradigm. In this paradigm, subjects shadowed a word target presented closely after an auditory memory list before they recalled the list. The phonological relationship between the shadowing target and the final item on the memory list was manipulated. Experiments 1 and 2 demonstrated that targets sounding similar to the list-final memory item generally took longer to shadow than unrelated targets. This inhibitory effect of phonological relatedness was more pronounced with tense- than lax-vowel pseudoword recall lists. The interaction between vowel tenseness and phonological relatedness was replicated in Experiment 3 using shorter lists of real words. In Experiment 4, concurrent articulation was applied during list learning to block rehearsal; consequently, neither the phonological relatedness effect nor its interaction with vowel tenseness emerged. Experiments 5 and 6 manipulated the occurrence frequencies and lexicality of the recall items, respectively, instead of vowel tenseness. Unlike vowel tenseness, these non-articulatory memory factors failed to interact with the phonological relatedness effect. Experiment 7 orthogonally manipulated the vowel tenseness and frequencies of the recall items; slowing in shadowing times due to phonological relatedness was modulated by vowel tenseness but not frequency. Taken together, these results suggest that under the present paradigm, the modifying effect of vowel tenseness on the magnitude of slowing in shadowing due to phonological relatedness is indicative of a prominent articulatory component in verbal short-term retention. The shadowing-plus-recall approach avoids confounding overt recall into internal memory processing, which is an inherent problem of the traditional immediate serial recall and span tasks.     

Feature assignment in perception of auditory figure

Because the environment often includes multiple sounds that overlap in time, listeners must segregate a sound of interest (the auditory figure) from other co-occurring sounds (the unattended auditory ground). We conducted a series of experiments to clarify the principles governing the extraction of auditory figures. We distinguish between auditory "objects" (relatively punctate events, such as a dog's bark) and auditory "streams" (sounds involving a pattern over time, such as a galloping rhythm). In Experiments 1 and 2, on each trial 2 sounds-an object (a vowel) and a stream (a series of tones)-were presented with 1 target feature that could be perceptually grouped with either source. In each block of these experiments, listeners were required to attend to 1 of the 2 sounds, and report its perceived category. Across several experimental manipulations, listeners were more likely to allocate the feature to an impoverished object if the result of the grouping was a good, identifiable object. Perception of objects was quite sensitive to feature variation (noise masking), whereas perception of streams was more robust to feature variation. In Experiment 3, the number of sound sources competing for the feature was increased to 3. This produced a shift toward relying more on spatial cues than on the potential contribution of the feature to an object's perceptual quality. The results support a distinction between auditory objects and streams, and provide new information about the way that the auditory world is parsed.     

Coherence masking protection in speech sounds: The role of formant synchrony

Three experiments were performed to examine listeners’ thresholds for identifying stimuli whose spectra were modeled after the vowels /i/ and /ε/, with the differences between these stimuli restricted to the frequency of the first formant. The stimuli were presented in a low-pass masking noise that spectrally overlapped the first formant but not the higher formants. Identification thresholds were lower when the higher formants were present than when they were not, even though the first formant contained the only distinctive information for stimulus identification. This indicates that listeners were more sensitive in identifying the first formant energy through its contribution to the vowel than as an independent percept; this effect is given the namecoherence masking protection. The first experiment showed this effect for synthetic vowels in which the distinctive first formant was supported by a series of harmonics that progressed through the higher formants. In the second two experiments, the harmonics in the first formant region were removed, and the first formant was simulated by a narrow band of noise. This was done so that harmonic relations did not provide a basis for grouping the lower formant with the higher formants; coherence masking protection was still observed. However, when the temporal alignment of the onsets and offsets of the higher and lower formants was disrupted, the effect was eliminated, although the stimuli were still perceived as vowels. These results are interpreted as indicating that general principles of auditory grouping that can exploit regularities in temporal patterns cause acoustic energy belonging to a coherent speech sound to stand out in the auditory scene.     

Auditory grouping based on fundamental frequency and formant peak frequency

The perceptual grouping of a four-tone cycle was studied as a function of differences in fundamental frequencies and the frequencies of spectral peaks. Each tone had a single formant and at least 13 harmonics. In Experiment 1 the formant was created by filtering a flat spectrum and in Experiment 2 by adding harmonics. Fundamental frequency was found to be capable of controlling grouping even when the spectra spanned exactly the same frequency range. Formant peak separation became more effective as the sharpness (amplitude of the peak relative to a spectral pedestal) increased. The effect of each type of acoustic difference depended on the task. Listeners could group the tones by either sort of difference but were also capable of resisting the disruptive effect of the other one. This was taken as evidence for the presence of a schema-based process of perceptual grouping and the relative weakness of primitive segregation.     

The perceptual segregation of simultaneous auditory signals: pulse train segregation and vowel segregation

In the experiments reported here, we attempted to find out more about how the auditory system is able to separate two simultaneous harmonic sounds. Previous research (Halikia & Bregman, 1984a, 1984b; Scheffers, 1983a) had indicated that a difference in fundamental frequency (F0) between two simultaneous vowel sounds improves their separate identification. In the present experiments, we looked at the effect of F0s that changed as a function of time. In Experiment 1, pairs of unfiltered or filtered pulse trains were used. Some were steady-state, and others had gliding F0s; different F0 separations were also used. The subjects had to indicate whether they had heard one or two sounds. The results showed that increased F0 differences and gliding F0s facilitated the perceptual separation of simultaneous sounds. In Experiments 2 and 3, simultaneous synthesized vowels were used on frequency contours that were steady-state, gliding in parallel (parallel glides), or gliding in opposite directions (crossing glides). The results showed that crossing glides led to significantly better vowel identification than did steady-state F0s. Also, in certain cases, crossing glides were more effective than parallel glides. The superior effect of the crossing glides could be due to the common frequency modulation of the harmonics within each component of the vowel pair and the consequent decorrelation of the harmonics between the two simultaneous vowels.     

Specialist musical training and the octave illusion: analytical listening and veridical perception by pipe organists

The octave illusion is a useful tool for investigation of the contribution of specialist training to auditory perception. The stimulus that induces the illusion involves two tones with a frequency ratio of 2:1, presented dichotically, and with ear of presentation reversed every 250 ms. Most listeners report hearing a single tone that alternates from high in the right ear to low in the left ear [Scientific American 233 (1975) 92-104]. The first experiment investigated the hypothesis that musical training contributes to veridical perception of an ambiguous stimulus. As hypothesized, participants with the highest level of musical training were more likely to perceive the stimulus veridically. Exploring the effects of specialist training, Experiment 2 contrasted expert pipe organists with other instrumentalists. As hypothesized, participants expert in playing pipe organ--an instrument with harmonic and spatial features similar to those of the octave illusion--were more likely to perceive the stimulus veridically. The results have implications for plasticity of the auditory system and the analytical listening that accompanies specialist, intensive training and rehearsal.