Feature parsing: feature cue mapping in spoken word recognition

For listeners to recognize words, they must map temporally distributed phonetic feature cues onto higher order phonological representations. Three experiments are reported that were performed to examine what information listeners extract from assimilated segments (e.g., place-assimilated tokens of cone that resemble comb) and how they interpret it. Experiment 1 employed form priming to demonstrate that listeners activate the underlying form of CONE, but not of its neighbor (COMB). Experiment 2 employed phoneme monitoring to show that the same assimilated tokens facilitate the perception of postassimilation context. Together, the results of these two experiments suggest that listeners recover both the underlying place of the modified item and information about the subsequent item from the same modified segment. Experiment 3 replicated Experiment 1, using different postassimilation contexts to demonstrate that context effects do not reflect familiarity with a given assimilation process. The results are discussed in the context of general auditory grouping mechanisms.     

Imitation in shadowing words

Imitation of shadowed words was evaluated using Goldinger’s (1998) AXB paradigm. The first experiment was a replication of Goldinger’s experiments with different tokens. Experiment 1’s AXB tests showed that shadowed words were judged to be better imitations of target words than were baseline (read) counterparts more often than chance (.50). Order of presentation of baseline and shadowed words in the AXB test also significantly influenced judgments. Degree of prior exposure to token words did not significantly influence judgments of imitation. Experiment 2 employed modified target tokens with extended voice onset times (VOTs). In addition to AXB tests, VOTs of response tokens were compared across baseline and shadowing conditions. The AXB tests revealed shadowed words to be better imitations of target tokens than baseline, without an influence of AXB presentation order. Differences between baseline and shadowing VOTs were greater when VOTs were extended. The implications of spontaneous imitation in nonsocial settings are considered.     

Commonsense beliefs and the identification of familiar voices

Two experiments are reported in which participants attempted to reject the tape‐recorded voice of a stranger and identify by name the voices of three personal associates who differed in their level of familiarity. In Experiment 1 listeners were asked to identify speakers as soon as possible, but were not allowed to change their responses once made. In Experiment 2 listeners were permitted to change their responses over successive presentations of increasing durations of voice segments. Also, in Experiment 2 half of the listeners attempted to identify speakers who spoke in normal‐tone voices, and the remainder attempted to identify the same speakers who spoke in whispers. Separate groups of undergraduate students attempted to predict the performance of the listeners in both experiments. Accuracy of performance depended on the familiarity of speakers and tone of speech. A between‐subjects analysis of rated confidence was diagnostic of accuracy for high familiar and low familiar speakers (Experiment 1), and for moderate familiar and unfamiliar normal‐tone speakers (Experiment 2). A modified between‐subjects analysis assessed across the four levels of familiarity yielded reliable accuracy‐confidence correlations in both experiments. Beliefs about the accuracy of voice identification were inflated relative to the significantly lower actual performance for most of the normal‐tone and whispered‐speech conditions. Forensic significance and generalizations are addressed. Copyright © 2001 John Wiley & Sons, Ltd.     

Acoustic,semantic and phonetic influences in spoken warning signal words

Three experiments are reported which explore the relationship between semantic, acoustic and phonetic variables in the judgement of eight warning signal words. Experiment 1 shows that listeners can distinguish very clearly between urgent and non‐urgent versions of the words when spoken by real speakers, and that some signal words such as ‘deadly’ and ‘danger’ score more highly than words such as ‘attention’ and ‘don't’. It also shows that the three dimensions of perceived urgency, appropriateness and believability of these words are highly correlated. Experiment 2 replicates Experiment 1 using synthesized voices where acoustic variables are controlled. The semantic effects are replicated, and to some extent appropriateness and believability are found to function differently from that of perceived urgency. Experiment 3 compares the same set of eight signal words with a set of phonetically similar neutral words, showing that warning signal words are rated significantly higher, and largely maintain their previous rank ordering. Copyright © 2003 John Wiley & Sons, Ltd.     

Recognizing prosody across modalities, face areas and speakers: examining perceivers' sensitivity to variable realizations of visual prosody

Prosody can be expressed not only by modification to the timing, stress and intonation of auditory speech but also by modifying visual speech. Studies have shown that the production of visual cues to prosody is highly variable (both within and across speakers), however behavioural studies have shown that perceivers can effectively use such visual cues. The latter result suggests that people are sensitive to the type of prosody expressed despite cue variability. The current study investigated the extent to which perceivers can match visual cues to prosody from different speakers and from different face regions. Participants were presented two pairs of sentences (consisting of the same segmental content) and were required to decide which pair had the same prosody. Experiment 1 tested visual and auditory cues from the same speaker and Experiment 2 from different speakers. Experiment 3 used visual cues from the upper and the lower face of the same talker and Experiment 4 from different speakers. The results showed that perceivers could accurately match prosody even when signals were produced by different speakers. Furthermore, perceivers were able to match the prosodic cues both within and across modalities regardless of the face area presented. This ability to match prosody from very different visual cues suggests that perceivers cope with variation in the production of visual prosody by flexibly mapping specific tokens to abstract prosodic types.     

The impact of Parkinson's disease on vocal-prosodic communication from the perspective of listeners

An expressive disturbance of speech prosody has long been associated with idiopathic Parkinson's disease (PD), but little is known about the impact of dysprosody on vocal-prosodic communication from the perspective of listeners. Recordings of healthy adults (n=12) and adults with mild to moderate PD (n=21) were elicited in four speech contexts in which prosody serves a primary function in linguistic or emotive communication (phonemic stress, contrastive stress, sentence mode, and emotional prosody). Twenty independent listeners naive to the disease status of individual speakers then judged the intended meanings conveyed by prosody for tokens recorded in each condition. Findings indicated that PD speakers were less successful at communicating stress distinctions, especially words produced with contrastive stress, which were identifiable to listeners. Listeners were also significantly less able to detect intended emotional qualities of Parkinsonian speech, especially for anger and disgust. Emotional expressions that were correctly recognized by listeners were consistently rated as less intense for the PD group. Utterances produced by PD speakers were frequently characterized as sounding sad or devoid of emotion entirely (neutral). Results argue that motor limitations on the vocal apparatus in PD produce serious and early negative repercussions on communication through prosody, which diminish the social-linguistic competence of Parkinsonian adults as judged by listeners.     

Laterality differences in speaker and consonant identification in dichotic listening

Two dichotic listening experiments assess the lateralization of speaker identification in right-handed native English speakers. Stimuli were tokens of /ba/, /da/, /pa/, and /ta/ pronounced by two male and two female speakers. In Experiment 1, subjects identified either the two consonants in dichotic stimuli spoken by the same person, or identified two speakers in dichotic tokens of the same syllable. In Experiment 2 new subjects identified the two consonants or the two speakers in pairs in which both consonant and speaker distinguished the pair members. Both experiments yielded significant right-ear advantages for consonant identification and nonsignificant ear differences for speaker identification. Fewer errors were made for speaker judgments than for consonant judgments, and for speaker judgments for pairs in which the speakers were of the same sex than for pairs in which speaker sex differed. It is concluded that, as in vowel identification, neither hemisphere clearly dominates in dichotic speaker identification, perhaps because of minor information loss in the ipsilateral pathways.     

Foreign accent strength and listener familiarity with an accent codetermine speed of perceptual adaptation

We investigated how the strength of a foreign accent and varying types of experience with foreign-accented speech influence the recognition of accented words. In Experiment 1, native Dutch listeners with limited or extensive prior experience with German-accented Dutch completed a cross-modal priming experiment with strongly, medium, and weakly accented words. Participants with limited experience were primed by the medium and weakly accented words, but not by the strongly accented words. Participants with extensive experience were primed by all accent types. In Experiments 2 and 3, Dutch listeners with limited experience listened to a short story before doing the cross-modal priming task. In Experiment 2, the story was spoken by the priming task speaker and either contained strongly accented words or did not. Strongly accented exposure led to immediate priming by novel strongly accented words, while exposure to the speaker without strongly accented tokens led to priming only in the experiment’s second half. In Experiment 3, listeners listened to the story with strongly accented words spoken by a different German-accented speaker. Listeners were primed by the strongly accented words, but again only in the experiment’s second half. Together, these results show that adaptation to foreign-accented speech is rapid but depends on accent strength and on listener familiarity with those strongly accented words.     

Relevance of the listener’s motor system in recalling phrases enacted by the speaker

Memory for series of action phrases improves in listeners when speakers accompany each phrase with congruent gestures compared to when speakers stay still. Studies reveal that the listeners’ motor system, at encoding, plays a crucial role in this enactment effect. We present two experiments on gesture observation, which explored the role of the listeners’ motor system at recall. The participants listened to the phrases uttered by a speaker in two conditions in each experiment. In the gesture condition, the speaker uttered the phrases with accompanying congruent gestures, and in the no-gesture condition, the speaker stayed still while uttering the phrases. The participants were then invited, in both conditions of the experiments, to perform a motor task while recalling the phrases proffered by the speaker. The results revealed that the advantage of observing gestures on memory disappears if the listeners move at recall arms and hands (same motor effectors moved by the speaker, Experiment 1a), but not when the listeners move legs and feet (different motor effectors from those moved by the speaker, Experiment 1b). The results suggest that the listeners’ motor system is involved not only during the encoding of action phrases uttered by a speaker but also when recalling these phrases during retrieval.     

Ease of processing constrains the activation flow in the conceptual-lexical system during speech planning

In 3 picture-word interference experiments, speakers named a target object in the presence of an unrelated not-to-be-named context object. Distractor words, which were phonologically related or unrelated to the context object's name, were used to determine whether the context object had become phonologically activated. All objects had high frequency names, and the ease of processing of these objects was manipulated by a visual degradation technique. In Experiment 1, both objects were nondegraded; in Experiment 2, both objects were degraded; and in Experiment 3, either the target object or the context object was degraded. Distractor words, which were phonologically related to the context objects, interfered with the naming response when both objects were nondegraded, indicating that the context objects had become phonologically coactivated. The effect vanished when both objects were degraded, when only the context object was degraded, and when only the target object was degraded. These data demonstrate that the amount of available processing resources constrains the forward cascading of activation in the conceptual-lexical system. Context objects are likely to become phonologically coactivated if they are easily retrieved and if prioritized target processing leaves sufficient resources.     

Effects of imagining speakers’ voices on the retention of words presented visually

Subjects were introduced to one male and one female voice by a tape recording with instructions to attend to characteristics of the voices. Then 18 pairs of words were presented visually on slides. The subject’s task during each 10-sec interslide interval was to repeat silently the pair of words over and over again in the male voice, in the female voice, or in the subject’s own voice. A surprise recognition test for the words indicated that the words were more likely to be recognized if they were spoken in the same Voice at test as was used to repeat them during presentation. Recognition of the words repeated in the subject’s own voice was not affected by the sex of the speaker at test. In Experiment 2, different speakers were used at test than those used by the subjects to repeat the words. The interaction between the sex of voice used at encoding and at test was again significant, but recognition was generally lower than in Experiment 1. It was concluded that it is not necessary to assume that subjects have literal copies of spoken words in memory but speaker’s voice does form an integral part of the verbal memory code and its influence is specific to a given speaker as well as to a given class of speakers (male or female).     

Speech planning during multiple-object naming: Effects of ageing

Two experiments were conducted with younger and older speakers. In Experiment 1, participants named single objects that were intact or visually degraded, while hearing distractor words that were phonologically related or unrelated to the object name. In both younger and older participants naming latencies were shorter for intact than for degraded objects and shorter when related than when unrelated distractors were presented. In Experiment 2, the single objects were replaced by object triplets, with the distractors being phonologically related to the first object's name. Naming latencies and gaze durations for the first object showed degradation and relatedness effects that were similar to those in single-object naming. Older participants were slower than younger participants when naming single objects and slower and less fluent on the second but not the first object when naming object triplets. The results of these experiments indicate that both younger and older speakers plan object names sequentially, but that older speakers use this planning strategy less efficiently.     

Imitation in shadowing words

Imitation of shadowed words was evaluated using Goldinger's (1998) AXB paradigm. The first experiment was a replication of Goldinger's experiments with different tokens. Experiment 1's AXB tests showed that shadowed words were judged to be better imitations of target words than were baseline (read) counterparts more often than chance (.50). Order of presentation of baseline and shadowed words in the AXB test also significantly influenced judgments. Degree of prior exposure to token words did not significantly influence judgments of imitation. Experiment 2 employed modified target tokens with extended voice onset times (VOTs). In addition to AXB tests, VOTs of response tokens were compared across baseline and shadowing conditions. The AXB tests revealed shadowed words to be better imitations of target tokens than baseline, without an influence of AXB presentation order. Differences between baseline and shadowing VOTs were greater when VOTs were extended. The implications of spontaneous imitation in nonsocial settings are considered.     

Speech planning during multiple-object naming: effects of ageing

Two experiments were conducted with younger and older speakers. In Experiment 1, participants named single objects that were intact or visually degraded, while hearing distractor words that were phonologically related or unrelated to the object name. In both younger and older participants naming latencies were shorter for intact than for degraded objects and shorter when related than when unrelated distractors were presented. In Experiment 2, the single objects were replaced by object triplets, with the distractors being phonologically related to the first object's name. Naming latencies and gaze durations for the first object showed degradation and relatedness effects that were similar to those in single-object naming. Older participants were slower than younger participants when naming single objects and slower and less fluent on the second but not the first object when naming object triplets. The results of these experiments indicate that both younger and older speakers plan object names sequentially, but that older speakers use this planning strategy less efficiently.     

Can hearing puter activate pupil? Phonological competition and the processing of reduced spoken words in spontaneous conversations

In listeners' daily communicative exchanges, they most often hear casual speech, in which words are often produced with fewer segments, rather than the careful speech used in most psycholinguistic experiments. Three experiments examined phonological competition during the recognition of reduced forms such as [pjut?r] for computer using a target-absent variant of the visual world paradigm. Listeners' eye movements were tracked upon hearing canonical and reduced forms as they looked at displays of four printed words. One of the words was phonologically similar to the canonical pronunciation of the target word, one word was similar to the reduced pronunciation, and two words served as unrelated distractors. When spoken targets were presented in isolation (Experiment 1) and in sentential contexts (Experiment 2), competition was modulated as a function of the target word form. When reduced targets were presented in sentential contexts, listeners were probabilistically more likely to first fixate reduced-form competitors before shifting their eye gaze to canonical-form competitors. Experiment 3, in which the original /p/ from [pjut?r] was replaced with a “real” onset /p/, showed an effect of cross-splicing in the late time window. We conjecture that these results fit best with the notion that speech reductions initially activate competitors that are similar to the phonological surface form of the reduction, but that listeners nevertheless can exploit fine phonetic detail to reconstruct strongly reduced forms to their canonical counterparts.     

The Differential Role of Syllabic Structure in Stem Completion for French and English

Four experiments were carried out to examine the role of a word's internal structure (i.e. syllables) in stem completion for French and English speakers. Subjects studied a series of unrelated words, selected so that two words shared their initial consonant-vowel-consonant (CVC) segment (e.g. BALANCE-BALCON). Subjects were then presented with CV or CVC stems (e.g. BA or BAL), half of which corresponded to the studied words' initial segment, and were asked to produce the first word that came to mind. Half the subjects performed the entire task in the auditory modality, half did so in the visual modality (Experiment 1). In both modalities, French subjects completed the stems more often with studied words in which the initial syllable matched the stem structure (e.g. BALCON for BAL) than with studied words that did not match (e.g. BALANCE for BAL). These syllabic effects were dissociable from explicit memory (Experiment 2) and appear to be language-specific, since they were obtained with French speakers but not with English speakers (Experiments 3 and 4). The results are highly consistent with the notion that implicit memory for words reflects the operations of perceptual phonological representations which are organised differently in French and English.     

Repetition blindness and illusory conjunctions: errors in binding visual types with visual tokens.

Repetition blindness (Kanwisher, 1986, 1987) has been defined as the failure to detect or recall repetitions of words presented in rapid serial visual presentation (RSVP). The experiments presented here suggest that repetition blindness (RB) is a more general visual phenomenon, and examine its relationship to feature integration theory (Treisman & Gelade, 1980). Experiment 1 shows RB for letters distributed through space, time, or both. Experiment 2 demonstrates RB for repeated colors in RSVP lists. In Experiments 3 and 4, RB was found for repeated letters and colors in spatial arrays. Experiment 5 provides evidence that the mental representations of discrete objects (called "visual tokens" here) that are necessary to detect visual repetitions (Kanwisher, 1987) are the same as the "object files" (Kahneman & Treisman, 1984) in which visual features are conjoined. In Experiment 6, repetition blindness for the second occurrence of a repeated letter resulted only when the first occurrence was attended to. The overall results suggest that a general dissociation between types and tokens in visual information processing can account for both repetition blindness and illusory conjunctions.     

视觉与听觉情绪信息关系判断中的交互作用

通过要求被试判断同时呈现的视听信息情绪效价的关系,考察视听情绪信息整合加工特点。实验一中词汇效价与韵律效价不冲突,实验二中词汇效价与韵律效价冲突。两个实验一致发现当面孔表情为积极时,被试对视听通道情绪信息关系判断更准确;实验二还发现,当面孔表情为消极时,相对于韵律线索,被试根据语义线索对视听信息关系判断更迅速。上述结果说明视听信息在同时呈现时,视觉信息可能先行加工,并影响到随后有关视听关系的加工。     

Categorization and normalization of vowels by 3-year-old children

A two-alternative, forced-choice procedure was used in two experiments to test 3-year-old children’s categorization of natural vowel tokens produced by several talkers. An appropriate pointing response (right or left) was visually reinforced with one of two television displays. In Experiment 1, the stimuli were isolated tokens of /a/ and /i/ produced by a male adult, a female adult, a male child, and a female child. In Experiment 2, the stimuli were isolated tokens of /æ/ and /∧/ produced by the same talkers. In both experiments, 3-year-olds spontaneously generalized their pointing responses from the male adult vowel tokens to the corresponding vowels produced by the other talkers. Children reinforced for an arbitrary grouping of the two vowel categories persisted in categorizing on the basis of vowel quality. Results from both experiments demonstrate the presence of perceptual constancy for vowel tokens across talkers. In particular, the results from Experiment 2 provide evidence for normalization of isolated, quasi-steady-state vowel tokens because the formant values for tokens of /∧/ produced by the woman and the two children were closer to the formant frequencies of the male adult’s /æ/ than to the male adult’s /∧/.