Examining the time course of indexical specificity effects in spoken word recognition

Variability in talker identity and speaking rate, commonly referred to as indexical variation, has demonstrable effects on the speed and accuracy of spoken word recognition. The present study examines the time course of indexical specificity effects to evaluate the hypothesis that such effects occur relatively late in the perceptual processing of spoken words. In 3 long-term repetition priming experiments, the authors examined reaction times to targets that were primed by stimuli that matched or mismatched on the indexical variable of interest (either talker identity or speaking rate). Each experiment was designed to manipulate the speed with which participants processed the stimuli. The results demonstrate that indexical variability affects participants' perception of spoken words only when processing is relatively slow and effortful.     

Famous talker effects in spoken word recognition

Previous work has demonstrated that talker-specific representations affect spoken word recognition relatively late during processing. However, participants in these studies were listening to unfamiliar talkers. In the present research, we used a long-term repetition-priming paradigm and a speeded-shadowing task and presented listeners with famous talkers. In Experiment 1, half the words were spoken by Barack Obama, and half by Hillary Clinton. Reaction times (RTs) to repeated words were shorter than those to unprimed words only when repeated by the same talker. However, in Experiment 2, using nonfamous talkers, RTs to repeated words were shorter than those to unprimed words both when repeated by the same talker and when repeated by a different talker. Taken together, the results demonstrate that talker-specific details can affect the perception of spoken words relatively early during processing when words are spoken by famous talkers.     

Hemispheric differences in specificity effects in talker identification

In the visual domain, Marsolek and colleagues (1999, 2008) have found support for two dissociable and parallel neural subsystems underlying object and shape recognition: an abstract-category subsystem that operates more effectively in the left cerebral hemisphere (LH), and a specific-exemplar subsystem that operates more effectively in the right cerebral hemisphere (RH). Evidence of this asymmetry has been observed in priming specificity for linguistic (words, pseudoword forms) and nonlinguistic (objects) stimuli. In the auditory domain, the authors previously found hemispheric asymmetries in priming effects for linguistic (spoken words) and nonlinguistic (environmental sounds) stimuli. In the present study, the same asymmetrical pattern was observed in talker identification by means of two long-term repetition-priming experiments. Both experiments consisted of a familiarization phase and a final talker identification test phase, using sentences as stimuli. The results showed that specificity effects (an advantage for same-sentence priming, relative to different-sentence priming) emerged when the target stimuli were presented to the left ear (RH), but not when the target stimuli were presented to the right ear (LH). Taken together, this consistent asymmetrical pattern of data from both domains-visual and auditory-may be indicative of a more general property of the human perceptual processing system. Theoretical implications are discussed.     

Phonological priming in spoken word recognition: Task effects

In two experiments, we examined the role of phonological relatedness between spoken items using both the lexical decision task and the shadowing task. In Experiment 1, words were used as primes and overlaps of zero (control), one, two, or all four or five (repetition) phonemes were compared. Except for the repetition conditions, in which facilitation was found, phonological overlap resulted in interference on word responses. These effects occurred in both tasks but were larger in lexical decision than in shadowing. The effects that were evident in shadowing can be attributed to an attentional mechanism linked to the subjects' expectancies of repetitions. The extra effects obtained in lexical decision can be interpreted by taking into account both activation of the response corresponding to the prime's lexical status and postlexical processes that check for phonological congruency between prime and target. In Experiment 2, some modifications were introduced to prevent the involvement of strategic factors, and pseudowords were used as primes. No effect at all was observed in shadowing, whereas in lexical decision interference effects occurred, which is consistent with the hypothesis that lexical decision may be negatively affected by finding a phonological discrepancy at the same time as the primed response is reactivated. Neither experiment provided evidence for the occurrence of phonological priming in the perceptual processing of words.     

Individual differences in online spoken word recognition: Implications for SLI

Thirty years of research has uncovered the broad principles that characterize spoken word processing across listeners. However, there have been few systematic investigations of individual differences. Such an investigation could help refine models of word recognition by indicating which processing parameters are likely to vary, and could also have important implications for work on language impairment. The present study begins to fill this gap by relating individual differences in overall language ability to variation in online word recognition processes. Using the visual world paradigm, we evaluated online spoken word recognition in adolescents who varied in both basic language abilities and non-verbal cognitive abilities. Eye movements to target, cohort and rhyme objects were monitored during spoken word recognition, as an index of lexical activation. Adolescents with poor language skills showed fewer looks to the target and more fixations to the cohort and rhyme competitors. These results were compared to a number of variants of the TRACE model (McClelland & Elman, 1986) that were constructed to test a range of theoretical approaches to language impairment: impairments at sensory and phonological levels; vocabulary size, and generalized slowing. None of the existing approaches were strongly supported, and variation in lexical decay offered the best fit. Thus, basic word recognition processes like lexical decay may offer a new way to characterize processing differences in language impairment.     

Sound specificity effects in spoken word recognition: The effect of integrality between words and sounds

Recent evidence has shown that nonlinguistic sounds co-occurring with spoken words may be retained in memory and affect later retrieval of the words. This sound-specificity effect shares many characteristics with the classic voice-specificity effect. In this study, we argue that the sound-specificity effect is conditional upon the context in which the word and sound coexist. Specifically, we argue that, besides co-occurrence, integrality between words and sounds is a crucial factor in the emergence of the effect. In two recognition-memory experiments, we compared the emergence of voice and sound specificity effects. In Experiment 1 , we examined two conditions where integrality is high. Namely, the classic voice-specificity effect (Exp. 1a) was compared with a condition in which the intensity envelope of a background sound was modulated along the intensity envelope of the accompanying spoken word (Exp. 1b). Results revealed a robust voice-specificity effect and, critically, a comparable sound-specificity effect: A change in the paired sound from exposure to test led to a decrease in word-recognition performance. In the second experiment, we sought to disentangle the contribution of integrality from a mere co-occurrence context effect by removing the intensity modulation. The absence of integrality led to the disappearance of the sound-specificity effect. Taken together, the results suggest that the assimilation of background sounds into memory cannot be reduced to a simple context effect. Rather, it is conditioned by the extent to which words and sounds are perceived as integral as opposed to distinct auditory objects.     

Immediate effects of form-class constraints on spoken word recognition

In many domains of cognitive processing there is strong support for bottom-up priority and delayed top-down (contextual) integration. We ask whether this applies to supra-lexical context that could potentially constrain lexical access. Previous findings of early context integration in word recognition have typically used constraints that can be linked to pair-wise conceptual relations between words. Using an artificial lexicon, we found immediate integration of syntactic expectations based on pragmatic constraints linked to syntactic categories rather than words: phonologically similar "nouns" and "adjectives" did not compete when a combination of syntactic and visual information strongly predicted form class. These results suggest that predictive context is integrated continuously, and that previous findings supporting delayed context integration stem from weak contexts rather than delayed integration.     

Hemispheric specialization in reading and word recognition

Three experiments dealing with hemispheric specialization are presented. In Experiment 1, words and/or faces were presented tachistoscopically to the left or right of fixation. Words were more accurately identified in the right visual field and faces were more accurately identified in the left visual field. A forced choice error analysis for words indicated that errors made for word stimuli were most frequently visually similar words and this effect was particularly pronounced in the left visual field. Two additional experiments supported this finding. On the basis of the results, it was argued that word identification is a multistage process, with visual feature analysis carried out by the right hemisphere and identification and naming by the left hemisphere. In addition, Kinsbourne's attentional model of brain function was rejected in favor of an anatomical model which suggests that simultaneous processing of verbal and nonverbal information does not constrict the attention of either hemisphere.     

The gender-marking effect in spoken word recognition

In this paper we examine whether the recognition of a spoken noun is affected by the gender marking—masculine or feminine—that is carried by a preceding word. In the first of two experiments, the gating paradigm was used to study the access of French nouns that were preceded by an appropriate gender marking, carried by an article, or preceded by no gender marking. In the second experiment, subjects were asked to make a lexical decision on the same material. A very strong facilitatory effect was found in both cases. The origin of the gender-marking effect is discussed, as well as the level of processing involved—lexical or syntactic.     

Grammatical gender in spoken word recognition in German

The role of grammatical gender for auditory word recognition in German was investigated in three experiments and two sets of corpus analyses. In the corpus analyses, gender information reduced the lexical search space as well as the amount of input needed to uniquely identify a word. To test whether this holds for on-line processing, two auditory lexical decision experiments (Experiments 1 and 3) were conducted using valid, invalid, or noise-masked articles as primes. Clear gender-priming effects were obtained in both experiments. Experiment 2 used phoneme monitoring with words and with pseudowords deviating from base words in one or more phonological features. Contrary to the lexical decision latencies, phoneme-monitoring latencies showed no influence of gender but did show similarity mismatch effects. We argue that gender information is not utilized early during word recognition. Rather, the presence of a valid article increases the initial familiarity of a word, facilitating subsequent responses.     

Cross-modal source information and spoken word recognition

In a cross-modal matching task, participants were asked to match visual and auditory displays of speech based on the identity of the speaker. The present investigation used this task with acoustically transformed speech to examine the properties of sound that can convey cross-modal information. Word recognition performance was also measured under the same transformations. The authors found that cross-modal matching was only possible under transformations that preserved the relative spectral and temporal patterns of formant frequencies. In addition, cross-modal matching was only possible under the same conditions that yielded robust word recognition performance. The results are consistent with the hypothesis that acoustic and optical displays of speech simultaneously carry articulatory information about both the underlying linguistic message and indexical properties of the talker.     

Frequency effects in spoken and visual word recognition: evidence from dual-task methodologies

The authors report 3 dual-task experiments concerning the locus of frequency effects in word recognition. In all experiments, Task 1 entailed a simple perceptual choice and Task 2 involved lexical decision. In Experiment 1, an underadditive effect of word frequency arose for spoken words. Experiment 2 also showed underadditivity for visual lexical decision. It was concluded that word frequency exerts an influence prior to any dual-task bottleneck. A related finding in similar dual-task experiments is Task 2 response postponement at short stimulus onset asynchronies. This was explored in Experiment 3, and it was shown that response postponement was equivalent for both spoken and visual word recognition. These results imply that frequency-sensitive processes operate early and automatically.     

The nature of phoneme representation in spoken word recognition

Four experiments used the psychological refractory period logic to examine whether integration of multiple sources of phonemic information has a decisional locus. All experiments made use of a dual-task paradigm in which participants made forced-choice color categorization (Task 1) and phoneme categorization (Task 2) decisions at varying stimulus onset asynchronies. In Experiment 1, Task 2 difficulty was manipulated using words containing matching or mismatching coarticulatory cues to the final consonant. The results showed that difficulty and onset asynchrony combined in an underadditive way, suggesting that the phonemic mismatch was resolved prior to a central decisional bottleneck. Similar results were found in Experiment 2 using nonwords. In Experiment 3, the manipulation of task difficulty involved lexical status, which once again revealed an underadditive pattern of response times. Finally, Experiment 4 compared this prebottleneck variable with a decisional variable: response key bias. The latter showed an additive pattern of responses. The experiments show that resolution of phonemic ambiguity can take advantage of cognitive slack time at short asynchronies, indicating that phonemic integration takes place at a relatively early stage of spoken word recognition.     

Eye movements during spoken word recognition in Russian children

This study explores incremental processing in spoken word recognition in Russian 5- and 6-year-olds and adults using free-viewing eye-tracking. Participants viewed scenes containing pictures of four familiar objects and clicked on a target embedded in a spoken instruction. In the cohort condition, two object names shared identical three-phoneme onsets. In the noncohort condition, all object names had unique onsets. Coarse-grain analyses of eye movements indicated that adults produced looks to the competitor on significantly more cohort trials than on noncohort trials, whereas children surprisingly failed to demonstrate cohort competition due to widespread exploratory eye movements across conditions. Fine-grain analyses, in contrast, showed a similar time course of eye movements across children and adults, but with cohort competition lingering more than 1s longer in children. The dissociation between coarse-grain and fine-grain eye movements indicates a need to consider multiple behavioral measures in making developmental comparisons in language processing.     

Context and spoken word recognition in a novel lexicon

Three eye movement studies with novel lexicons investigated the role of semantic context in spoken word recognition, contrasting 3 models: restrictive access, access-selection, and continuous integration. Actions directed at novel shapes caused changes in motion (e.g., looming, spinning) or state (e.g., color, texture). Across the experiments, novel names for the actions and the shapes varied in frequency, cohort density, and whether the cohorts referred to actions (Experiment 1) or shapes with action-congruent or action-incongruent affordances (Experiments 2 and 3). Experiment 1 demonstrated effects of frequency and cohort competition from both displayed and non-displayed competitors. In Experiment 2, a biasing context induced an increase in anticipatory eye movements to congruent referents and reduced the probability of looks to incongruent cohorts, without the delay predicted by access-selection models. In Experiment 3, context did not reduce competition from non-displayed incompatible neighbors as predicted by restrictive access models. The authors conclude that the results are most consistent with continuous integration models.     

Influences of spoken word planning on speech recognition

In 4 chronometric experiments, influences of spoken word planning on speech recognition were examined. Participants were shown pictures while hearing a tone or a spoken word presented shortly after picture onset. When a spoken word was presented, participants indicated whether it contained a prespecified phoneme. When the tone was presented, they indicated whether the picture name contained the phoneme (Experiment 1) or they named the picture (Experiment 2). Phoneme monitoring latencies for the spoken words were shorter when the picture name contained the prespecified phoneme compared with when it did not. Priming of phoneme monitoring was also obtained when the phoneme was part of spoken nonwords (Experiment 3). However, no priming of phoneme monitoring was obtained when the pictures required no response in the experiment, regardless of monitoring latency (Experiment 4). These results provide evidence that an internal phonological pathway runs from spoken word planning to speech recognition and that active phonological encoding is a precondition for engaging the pathway.     

Exploring the cohort model of spoken word recognition

Predictions derived from the Cohort Model of spoken word recognition were tested in four experiments using an auditory lexical decision task. The first experiment produced results that were compatible with the model, in that the point at which a word could be uniquely identified appeared to influence reaction times. The second and third experiments demonstrated that the processing of a nonword phoneme string continues after the point at which there are no possible continuations that would make a word. The number of phonemes following the point of deviation from a word was shown to affect reaction times, as well as the similarity of the nonword to a word. The final experiment demonstrated a frequency effect when high and low frequency words were matched on their point of unique identity. These last three results are not consistent with the Cohort Model and so an alternative account is put forward. According to this account, the first few phonemes are used to activate all words beginning with those phonemes and then these candidates are checked back to the original stimulus. This model provides greater flexibility than the Cohort Model and allows for mispronounced and misperceived words to be correctly recognized.     

Grammatical context constrains lexical competition in spoken word recognition

When perceiving spoken language, listeners must match the incoming acoustic phonetic input to lexical representations in memory. Models that quantify this process propose that the input activates multiple lexical representations in parallel and that these activated representations compete for recognition (Weber & Scharenborg, 2012). In two experiments, we assessed how grammatically constraining contexts alter the process of lexical competition. The results suggest that grammatical context constrains the lexical candidates that are activated to grammatically appropriate competitors. Stimulus words with little competition from items of the same grammatical class benefit more from the addition of grammatical context than do words with more within-class competition. The results provide evidence that top-down contextual information is integrated in the early stages of word recognition. We propose adding a grammatical class level of analysis to existing models of word recognition to account for these findings.