Theories of spoken word recognition deficits in aphasia: evidence from eye-tracking and computational modeling

We used eye-tracking to investigate lexical processing in aphasic participants by examining the fixation time course for rhyme (e.g., carrot-parrot) and cohort (e.g., beaker-beetle) competitors. Broca’s aphasic participants exhibited larger rhyme competition effects than age-matched controls. A re-analysis of previously reported data (Yee, Blumstein, & Sedivy, 2008) confirmed that Wernicke’s aphasic participants exhibited larger cohort competition effects. Individual-level analyses revealed a negative correlation between rhyme and cohort competition effect size across both groups of aphasic participants. Computational model simulations were performed to examine which of several accounts of lexical processing deficits in aphasia might account for the observed effects. Simulation results revealed that slower deactivation of lexical competitors could account for increased cohort competition in Wernicke’s aphasic participants; auditory perceptual impairment could account for increased rhyme competition in Broca’s aphasic participants; and a perturbation of a parameter controlling selection among competing alternatives could account for both patterns, as well as the correlation between the effects. In light of these simulation results, we discuss theoretical accounts that have the potential to explain the dynamics of spoken word recognition in aphasia and the possible roles of anterior and posterior brain regions in lexical processing and cognitive control.     

Heeding the voice of experience: the role of talker variation in lexical access

Two experiments used the head-mounted eye-tracking methodology to examine the time course of lexical activation in the face of a non-phonemic cue, talker variation. We found that lexical competition was attenuated by consistent talker differences between words that would otherwise be lexical competitors. In Experiment 1, some English cohort word-pairs were consistently spoken by a single talker (male couch, male cows), while other word-pairs were spoken by different talkers (male sheep, female sheet). After repeated instances of talker-word pairings, words from different-talker pairs showed smaller proportions of competitor fixations than words from same-talker pairs. In Experiment 2, participants learned to identify black-and-white shapes from novel labels spoken by one of two talkers. All of the 16 novel labels were VCVCV word-forms atypical of, but not phonologically illegal in, English. Again, a word was consistently spoken by one talker, and its cohort or rhyme competitor was consistently spoken either by that same talker (same-talker competitor) or the other talker (different-talker competitor). Targets with different-talker cohorts received greater fixation proportions than targets with same-talker cohorts, while the reverse was true for fixations to cohort competitors; there were fewer erroneous selections of competitor referents for different-talker competitors than same-talker competitors. Overall, these results support a view of the lexicon in which entries contain extra-phonemic information. Extensions of the artificial lexicon paradigm and developmental implications are discussed.     

Studying the grammatical aspects of word recognition: lexical priming,parsing, and syntactic ambiguity resolution

Two experiments are reported examining the relationship between lexical and syntactic processing during language comprehension, combining techniques common to the on-line study of syntactic ambiguity resolution with priming techniques common to the study of lexical processing. By manipulating grammatical properties of lexical primes, we explore how lexically based knowledge is activated and guides combinatory sentence processing. Particularly, we find that nouns (like verbs, see Trueswell & Kim, 1998) can activate detailed lexically specific syntactic information and that these representations guide the resolution of relevant syntactic ambiguities pertaining to verb argument structure. These findings suggest that certain principles of knowledge representation common to theories of lexical knowledge—such as overlapping and distributed representations—also characterize grammatical knowledge. Additionally, observations from an auditory comprehension study suggest similar conclusions about the lexical nature of parsing in spoken language comprehension. They also suggest that thematic role and syntactic preferences are activated during word recognition and that both influence combinatory processing.     

Auditory lexical decisions in children with specific language impairment

This study examined spoken‐word recognition in children with specific language impairment (SLI) and normally developing children matched separately for age and receptive language ability. Accuracy and reaction times on an auditory lexical decision task were compared. Children with SLI were less accurate than both control groups. Two subgroups of children with SLI, distinguished by performance accuracy only, were identified. One group performed within normal limits, while a second group was significantly less accurate. Children with SLI were not slower than the age‐matched controls or language‐matched controls. Further, the time taken to detect an auditory signal, make a decision, or initiate a verbal response did not account for the differences between the groups. The findings are interpreted as evidence for language‐appropriate processing skills acting upon imprecise or underspecified stored representations.     

Examining the effects of variation in emotional tone of voice on spoken word recognition

Emotional tone of voice (ETV) is essential for optimal verbal communication. Research has found that the impact of variation in nonlinguistic features of speech on spoken word recognition differs according to a time course. In the current study, we investigated whether intratalker variation in ETV follows the same time course in two long-term repetition priming experiments. We found that intratalker variability in ETVs affected reaction times to spoken words only when processing was relatively slow and difficult, not when processing was relatively fast and easy. These results provide evidence for the use of both abstract and episodic lexical representations for processing within-talker variability in ETV, depending on the time course of spoken word recognition.     

Does input influence uptake? Links between maternal talk,processing speed and vocabulary size in Spanish‐learning children

It is well established that variation in caregivers’ speech is associated with language outcomes, yet little is known about the learning principles that mediate these effects. This longitudinal study (n = 27) explores whether Spanish‐learning children's early experiences with language predict efficiency in real‐time comprehension and vocabulary learning. Measures of mothers’ speech at 18 months were examined in relation to children's speech processing efficiency and reported vocabulary at 18 and 24 months. Children of mothers who provided more input at 18 months knew more words and were faster in word recognition at 24 months. Moreover, multiple regression analyses indicated that the influences of caregiver speech on speed of word recognition and vocabulary were largely overlapping. This study provides the first evidence that input shapes children's lexical processing efficiency and that vocabulary growth and increasing facility in spoken word comprehension work together to support the uptake of the information that rich input affords the young language learner.     

Electrophysiological responses to coarticulatory and word level miscues

The influence of coarticulation cues on spoken word recognition is not yet well understood. This acoustic/phonetic variation may be processed early and recognized as sensory noise to be stripped away, or it may influence processing at a later prelexical stage. The present study used event-related potentials (ERPs) in a picture/spoken word matching paradigm to examine the temporal dynamics of stimuli systematically violating expectations at three levels: entire word (lexical), initial phoneme (phonemic), or in coarticulation cues contained in the initial phoneme (subphonemic). We found that both coarticulatory and phonemic mismatches resulted in increased negativity in the N280, interpreted as indexing prelexical processing of subphonemic information. Further analyses revealed that the point of uniqueness differentially modulated subsequent early or late negativity depending on whether the first or second segment matched expectations, respectively. Finally, it was found that word-level but not coarticulatory mismatches modulated the later-going N400 component, indicating that subphonemic information does not influence word-level selection provided no lexical change has occurred. The results indicate that acoustic/phonetic variation resulting from coarticulation is preserved in and influences spoken word recognition as it becomes available, particularly during prelexical processing.     

Exploring the Mental Lexicon

This paper is a broad survey of issues that I have been examining within the domain of lexical access. Experiments are briefly outlined looking at the questions of morphological processing in both visual and spoken word recognition, phonological recoding in visual word recognition, orthographic influences in spoken word recognition, and a morphophonemic level of word representation. These issues are discussed in the framework of a model of lexical access where the recognition system and the production system are seen as separate representational systems.     

The interaction of meaning and sound in spoken word recognition

Models of spoken word recognition vary in the ways in which they capture the relationship between speech input and meaning. Modular accounts prohibit a word’s meaning from affecting the computation of its form-based representation, whereas interactive models allow activation at the semantic level to affect phonological processing. We tested these competing hypotheses by manipulating word familiarity and imageability, using lexical decision and repetition tasks. Responses to high-imageability words were significantly faster than those to low-imageability words. Repetition latencies were also analyzed as a function of cohort variables, revealing a significant imageability effect only for words that were members of large cohorts, suggesting that when the mapping from phonology to semantics is difficult, semantic information can help the discrimination process. Thus, these data support interactive models of spoken word recognition.     

Spoken Language Activation Alters Subsequent Sign Language Activation in L2 Learners of American Sign Language

A large body of literature has characterized unimodal monolingual and bilingual lexicons and how neighborhood density affects lexical access; however there have been relatively fewer studies that generalize these findings to bimodal (M2) second language (L2) learners of sign languages. The goal of the current study was to investigate parallel language activation in M2L2 learners of sign language and to characterize the influence of spoken language and sign language neighborhood density on the activation of ASL signs. A priming paradigm was used in which the neighbors of the sign target were activated with a spoken English word and compared the activation of the targets in sparse and dense neighborhoods. Neighborhood density effects in auditory primed lexical decision task were then compared to previous reports of native deaf signers who were only processing sign language. Results indicated reversed neighborhood density effects in M2L2 learners relative to those in deaf signers such that there were inhibitory effects of handshape density and facilitatory effects of location density. Additionally, increased inhibition for signs in dense handshape neighborhoods was greater for high proficiency L2 learners. These findings support recent models of the hearing bimodal bilingual lexicon, which posit lateral links between spoken language and sign language lexical representations.     

Eye Movements and Lexical Access in Spoken-Language Comprehension: Evaluating a Linking Hypothesis between Fixations and Linguistic Processing

A growing number of researchers in the sentence processing community are using eye movements to address issues in spoken language comprehension. Experiments using this paradigm have shown that visually presented referential information, including properties of referents relevant to specific actions, influences even the earliest moments of syntactic processing. Methodological concerns about task-specific strategies and the linking hypothesis between eye movements and linguistic processing are identified and discussed. These concerns are addressed in a review of recent studies of spoken word recognition which introduce and evaluate a detailed linking hypothesis between eye movements and lexical access. The results provide evidence about the time course of lexical activation that resolves some important theoretical issues in spoken-word recognition. They also demonstrate that fixations are sensitive to properties of the normal language-processing system that cannot be attributed to task-specific strategies.     

Hemispheric differences in indexical specificity effects in spoken word recognition

Variability in talker identity, one type of indexical variation, has demonstrable effects on the speed and accuracy of spoken word recognition. Furthermore, neuropsychological evidence suggests that indexical and linguistic information may be represented and processed differently in the 2 cerebral hemispheres, and is consistent with findings from the visual domain. For example, in visual word recognition, changes in font affect processing differently depending on which hemisphere initially processes the input. The present study examined whether hemispheric differences exist in spoken language as well. In 4 long-term repetition-priming experiments, the authors examined responses to stimuli that were primed by stimuli that matched or mismatched in talker identity. The results demonstrate that indexical variability can affect participants' perception of spoken words differently in the 2 hemispheres.     

The developmental turnpoint of orthographic consistency effects in speech recognition

The influence of orthography on children's online auditory word recognition was studied from the end of Grade 4 to the end of Grade 9 by examining the orthographic consistency effect in auditory lexical decision. Fourth-graders showed evidence of a widespread influence of orthography in their spoken word recognition system; words with rimes that can be spelled in two different ways (inconsistent) produced longer auditory lexical decision times and more errors than did consistent words. A similar consistency effect was also observed on pseudowords. With adult listeners, on exactly the same material, we replicated the usual pattern of an orthographic consistency effect restricted to words in lexical decision. From Grade 6 onward, this adult pattern of orthographic effect on spoken recognition is already observable.     

Syllabic Effects in Italian Lexical Access

Two cross-modal priming experiments tested whether lexical access is constrained by syllabic structure in Italian. Results extend the available Italian data on the processing of stressed syllables showing that syllabic information restricts the set of candidates to those structurally consistent with the intended word (Experiment 1). Lexical access, however, takes place as soon as possible and it is not delayed till the incoming input corresponds to the first syllable of the word. And, the initial activated set includes candidates whose syllabic structure does not match the intended word (Experiment 2). The present data challenge the early hypothesis that in Romance languages syllables are the units for lexical access during spoken word recognition. The implications of the results for our understanding of the role of syllabic information in language processing are discussed.     

Where do dialectal effects on speech processing come from? Evidence from a cross-dialect investigation

Accented speech has been seen as an additional impediment for speech processing; it usually adds linguistic and cognitive load to the listener's task. In the current study we analyse where the processing costs of regional dialects come from, a question that has not been answered yet. We quantify the proficiency of Basque–Spanish bilinguals who have different native dialects of Basque on many dimensions and test for costs at each of three levels of processing–phonemic discrimination, word recognition, and semantic processing. The ability to discriminate a dialect-specific contrast is affected by a bilingual's linguistic background less than lexical access is, and an individual's difficulty in lexical access is correlated with basic discrimination problems. Once lexical access is achieved, dialectal variation has little impact on semantic processing. The results are discussed in terms of the presence or absence of correlations between different processing levels. The implications of the results are considered for how models of spoken word recognition handle dialectal variation.     

The influence of speech rate and accent on access and use of semantic information

Circumstances in which the speech input is presented in sub-optimal conditions generally lead to processing costs affecting spoken word recognition. The current study indicates that some processing demands imposed by listening to difficult speech can be mitigated by feedback from semantic knowledge. A set of lexical decision experiments examined how foreign accented speech and word duration impact access to semantic knowledge in spoken word recognition. Results indicate that when listeners process accented speech, the reliance on semantic information increases. Speech rate was not observed to influence semantic access, except in the setting in which unusually slow accented speech was presented. These findings support interactive activation models of spoken word recognition in which attention is modulated based on speech demands.     

A dissociative word-frequency X levels-of-processing interaction in episodic recognition and lexical decision tasks

The effects of levels-of-processing and word frequency were directly compared in three different memory tests. In the episodic recognition test, the subjects decided whether or not a word or a pronounceable nonword had been previously studied. In the two lexical decision tests with either pronounceable or unpronounceable nonwords as distractors, the subjects decided whether a test item was a word or a nonword. There were four main results: (1) in all three tests, reaction times (RTs) in response to studied words were faster if they had received semantic rather than rhyme processing during study; (2) in the episodic recognition test, RTs were faster for low- than for high-frequency words; in both lexical decision tests, RTs were faster for high- than for low-frequency words, though less so when the nonword distractors were unpronounceable; (3) prior study facilitated lexical decisions more in response to low- than to high-frequency words, thereby attenuating the word-frequency effect, but more so when the nonword distractors were pronounceable; (4) in the lexical decision test with pronounceable nonword distractors, relative to prior rhyme processing, prior semantic processing facilitated performance more for high- than for low-frequency words, whereas the opposite was the case in the episodic recognition test. Discussion focused on the relationship of these results to current views of the mechanisms by which (1) word frequency and depth of processing affect performance in implicit and explicit memory tests, and (2) repetition priming attenuates word-frequency effects for lexical decisions.     

Individual differences in visual word recognition: insights from the English Lexicon Project

Empirical work and models of visual word recognition have traditionally focused on group-level performance. Despite the emphasis on the prototypical reader, there is clear evidence that variation in reading skill modulates word recognition performance. In the present study, we examined differences among individuals who contributed to the English Lexicon Project (http://elexicon.wustl.edu), an online behavioral database containing nearly 4 million word recognition (speeded pronunciation and lexical decision) trials from over 1,200 participants. We observed considerable within- and between-session reliability across distinct sets of items, in terms of overall mean response time (RT), RT distributional characteristics, diffusion model parameters (Ratcliff, Gomez, & McKoon, 2004), and sensitivity to underlying lexical dimensions. This indicates reliably detectable individual differences in word recognition performance. In addition, higher vocabulary knowledge was associated with faster, more accurate word recognition performance, attenuated sensitivity to stimuli characteristics, and more efficient accumulation of information. Finally, in contrast to suggestions in the literature, we did not find evidence that individuals were trading-off their utilization of lexical and nonlexical information.     

Implicit statistical learning in language processing: Word predictability is the key

Fundamental learning abilities related to the implicit encoding of sequential structure have been postulated to underlie language acquisition and processing. However, there is very little direct evidence to date supporting such a link between implicit statistical learning and language. In three experiments using novel methods of assessing implicit learning and language abilities, we show that sensitivity to sequential structure – as measured by improvements to immediate memory span for structurally-consistent input sequences – is significantly correlated with the ability to use knowledge of word predictability to aid speech perception under degraded listening conditions. Importantly, the association remained even after controlling for participant performance on other cognitive tasks, including short-term and working memory, intelligence, attention and inhibition, and vocabulary knowledge. Thus, the evidence suggests that implicit learning abilities are essential for acquiring long-term knowledge of the sequential structure of language – i.e., knowledge of word predictability – and that individual differences on such abilities impact speech perception in everyday situations. These findings provide a new theoretical rationale linking basic learning phenomena to specific aspects of spoken language processing in adults, and may furthermore indicate new fruitful directions for investigating both typical and atypical language development.     

Influences of lexical tone and pitch on word recognition in bilingual infants

Infants' abilities to discriminate native and non-native phonemes have been extensively investigated in monolingual learners, demonstrating a transition from language-general to language-specific sensitivities over the first year after birth. However, these studies have mostly been limited to the study of vowels and consonants in monolingual learners. There is relatively little research on other types of phonetic segments, such as lexical tone, even though tone languages are very well represented across languages of the world. The goal of the present study is to investigate how Mandarin Chinese-English bilingual learners contend with non-phonemic pitch variation in English spoken word recognition. This is contrasted with their treatment of phonemic changes in lexical tone in Mandarin spoken word recognition. The experimental design was cross-sectional and three age-groups were sampled (7.5months, 9months and 11months). Results demonstrated limited generalization abilities at 7.5months, where infants only recognized words in English when matched in pitch and words in Mandarin that were matched in tone. At 9months, infants recognized words in Mandarin Chinese that matched in tone, but also falsely recognized words that contrasted in tone. At this age, infants also recognized English words whether they were matched or mismatched in pitch. By 11months, infants correctly recognized pitch-matched and - mismatched words in English but only recognized tonal matches in Mandarin Chinese.