Functional specificity of the visual word form area: general activation for words and symbols but specific network activation for words

The functional specificity of the brain region known as the Visual Word Form Area (VWFA) was examined using fMRI. We explored whether this area serves a general role in processing symbolic stimuli, rather than being selective for the processing of words. Brain activity was measured during a visual 1-back task to English words, meaningful symbols (e.g., $, %), digits, words in an unfamiliar language (Hebrew), and geometric control stimuli. Mean activity in the functionally defined VWFA, as well as a pattern of whole-brain activity identified using a multivariate technique, did not differ for words and symbols, but was distinguished from that seen with other stimuli. However, functional connectivity analysis of this region identified a network of regions that was specific to words, including the left hippocampus, left lateral temporal, and left prefrontal cortex. Results support the hypothesis that activity in the VWFA plays a general role in processing abstract stimuli; however, the left VWFA is part of a unique network of brain regions active only during the word condition. These findings suggest that it is the neural "context" of the VWFA, i.e., the broader activity distributed in the brain that is correlated with VWFA, that is specific for visual word representation, not activity in this brain region per se.     

Speech segmentation is facilitated by visual cues

Evidence from infant studies indicates that language learning can be facilitated by multimodal cues. We extended this observation to adult language learning by studying the effects of simultaneous visual cues (nonassociated object images) on speech segmentation performance. Our results indicate that segmentation of new words from a continuous speech stream is facilitated by simultaneous visual input that it is presented at or near syllables that exhibit the low transitional probability indicative of word boundaries. This indicates that temporal audio-visual contiguity helps in directing attention to word boundaries at the earliest stages of language learning. Off-boundary or arrhythmic picture sequences did not affect segmentation performance, suggesting that the language learning system can effectively disregard noninformative visual information. Detection of temporal contiguity between multimodal stimuli may be useful in both infants and second-language learners not only for facilitating speech segmentation, but also for detecting word–object relationships in natural environments.     

Neural localization of semantic context effects in electromagnetic and hemodynamic studies

Measures of electrical brain activity (event-related potentials, ERPs) have been useful in understanding language processing for several decades. Extant data suggest that the amplitude of the N400 component of the ERP is a general index of the ease or difficulty of retrieving stored conceptual knowledge associated with a word, which is dependent on both the stored representation itself, and the retrieval cues provided by the preceding context. Recordings from patients with brain damage, intracranial recordings, and magnetoencephalographic data implicate a (probably large portion of) the left temporal lobe as the largest source of the N400 semantic context effect, with a substantial but lesser contribution from the right temporal lobe. Event-related functional magnetic resonance (fMRI) studies using semantic context manipulations are dominated by observations of greater hemodynamic activity for incongruent sentence completions or semantically unrelated words than congruent or related words, consistent with the direction of the ERP effect. The locations of the hemodynamic effects show some variability across studies, but one commonly identified region is the left superior temporal gyrus, which is compatible with the electrophysiological results. A second commonly identified region in the fMRI studies is the left inferior frontal gyrus, which does not appear to make a substantial contribution to the N400 effect.     

Words in the brain's language

If the cortex is an associative memory, strongly connected cell assemblies will form when neurons in different cortical areas are frequently active at the same time. The cortical distributions of these assemblies must be a consequence of where in the cortex correlated neuronal activity occurred during learning. An assembly can be considered a functional unit exhibiting activity states such as full activation ("ignition") after appropriate sensory stimulation (possibly related to perception) and continuous reverberation of excitation within the assembly (a putative memory process). This has implications for cortical topographies and activity dynamics of cell assemblies forming during language acquisition, in particular for those representing words. Cortical topographies of assemblies should be related to aspects of the meaning of the words they represent, and physiological signs of cell assembly ignition should be followed by possible indicators of reverberation. The following postulates are discussed in detail: (1) assemblies representing phonological word forms are strongly lateralized and distributed over perisylvian cortices; (2) assemblies representing highly abstract words such as grammatical function words are also strongly lateralized and restricted to these perisylvian regions; (3) assemblies representing concrete content words include additional neurons in both hemispheres; (4) assemblies representing words referring to visual stimuli include neurons in visual cortices; and (5) assemblies representing words referring to actions include neurons in motor cortices. Two main sources of evidence are used to evaluate these proposals: (a) imaging studies focusing on localizing word processing in the brain, based on stimulus-triggered event-related potentials (ERPs), positron emission tomography (PET), and functional magnetic resonance imaging (fMRI), and (b) studies of the temporal dynamics of fast activity changes in the brain, as revealed by high-frequency responses recorded in the electroencephalogram (EEG) and magnetoencephalogram (MEG). These data provide evidence for processing differences between words and matched meaningless pseudowords, and between word classes, such as concrete content and abstract function words, and words evoking visual or motor associations. There is evidence for early word class-specific spreading of neuronal activity and for equally specific high-frequency responses occurring later. These results support a neurobiological model of language in the Hebbian tradition. Competing large-scale neuronal theories of language are discussed in light of the data summarized. Neurobiological perspectives on the problem of serial order of words in syntactic strings are considered in closing.     

Reversing spoken items--mind twisting not tongue twisting

Using 12 participants we conducted an fMRI study involving two tasks, word reversal and rhyme judgment, based on pairs of natural speech stimuli, to study the neural correlates of manipulating auditory imagery under taxing conditions. Both tasks engaged the left anterior superior temporal gyrus, reflecting previously established perceptual mechanisms. Engagement of the left inferior frontal gyrus in both tasks relative to baseline could only be revealed by applying small volume corrections to the region of interest, suggesting that phonological segmentation played only a minor role and providing further support for factorial dissociation of rhyming and segmentation in phonological awareness. Most importantly, subtraction of rhyme judgment from word reversal revealed activation of the parietal lobes bilaterally and the right inferior frontal cortex, suggesting that the dynamic manipulation of auditory imagery involved in mental reversal of words seems to engage mechanisms similar to those involved in visuospatial working memory and mental rotation. This suggests that reversing spoken items is a matter of mind twisting rather than tongue twisting and provides support for a link between language processing and manipulation of mental imagery.     

汉语言语产生的语音加工单元——基于音位的研究

言语产生的语音加工单元具有跨语言的特异性。在印欧语言中, 音位是语音加工的重要功能单元。音位指具体语言中能够区别意义的最小语音单位, 如“big”包含三个音位/b/, /i/, /g/。目前, 在汉语言语产生中, 对音位的研究较少。本项目拟采用事件相关电位技术, 对汉语言语产生中的音位加工进行探讨, 试图考察：在汉语言语产生中, 1)音位加工的心理现实性, 以及音位表征是否受第二语言、汉语拼音习得、拼音使用经验的影响?2)音位的加工机制是怎样的?具体而言, 音位加工的特异性、位置编码、组合方式、时间进程是怎样的?对这些问题的回答, 将有助于深化对汉语言语产生的认识, 为建立汉语言语产生计算模型提供基础; 为比较印欧语言与汉语在机制上的异同提供基础; 为制定汉语语音教育教学方法提供心理学依据。     

Temporal cortex activation during speech recognition: an optical topography study

Cortical activity during speech recognition was examined using optical topography (OT), a recently developed non-invasive technique. To assess relative changes in hemoglobin oxygenation, local changes in near-infrared light absorption were measured simultaneously from 44 points in both hemispheres. A dichotic listening paradigm was used in this experiment, in which target stimuli and non-target stimuli were presented to different ears. Subjects were asked to track targets and to press a button when targets shifted from one ear to the other. We compared three tasks: (i) a control task, in which a tone was used as the target; (ii) a repeat task, in which the target was one repeated sentence; (iii) a story task, in which the targets were continuous sentences of a story. The activity for the story task, compared with the repeat task, was localized in the left superior temporal cortex. Relative to the control task, we observed in this region a larger increase in oxyhemoglobin concentration and a decrease in deoxyhemoglobin concentration in the story task than those in the repeat task. These results suggest that the activity in the left temporal association area reflects the load of auditory, memory, and language information processing.     

名词与动词加工的ERP差异

采用词汇判断任务和记录ERP的方法,探讨了名词与动词之间的差异以及词汇的具体性对这种词性效应的影响。研究结果表明,在200~300ms、300~400ms（即N400）时窗,词性与具体性的交互作用在两侧电极上达到显著水平,在两半球上具体名词所诱发的ERP比具体动词更负。并且,在N400上词性与电极位置有显著的交互作用,名词与动词的差异主要存在于左、右半球的额叶和颞叶。本研究结果部分支持了语言表征的神经生物学理论     

Predicting language: MEG evidence for lexical preactivation

It is widely assumed that prediction plays a substantial role in language processing. However, despite numerous studies demonstrating that contextual information facilitates both syntactic and lexical–semantic processing, there exists no direct evidence pertaining to the neural correlates of the prediction process itself. Using magnetoencephalography (MEG), this study found that brain activity was modulated by whether or not a specific noun could be predicted, given a picture prime. Specifically, before the noun was presented, predictive contexts triggered enhanced activation in left mid-temporal cortex (implicated in lexical access), ventro-medial prefrontal cortex (previously associated with top-down processing), and visual cortex (hypothesized to index the preactivation of predicted form features), successively. This finding suggests that predictive language processing recruits a top-down network where predicted words are activated at different levels of representation, from more ‘abstract’ lexical–semantic representations in temporal cortex, all the way down to visual word form features. The same brain regions that exhibited enhanced activation for predictive contexts before the onset of the noun showed effects of congruence during the target word. To our knowledge, this study is one of the first to directly investigate the anticipatory stage of predictive language processing.     

An fMRI investigation of covertly and overtly produced mono- and multisyllabic words

Studies suggest that the left insula may play an important role in speech motor programming. We used functional magnetic resonance imaging to investigate the role of the left insula in the production of monosyllabic or multisyllabic words during overt and covert speech conditions. The left insula did not show a BOLD response for multisyllabic words (which should require more speech motor programming) that was different from that for monosyllabic words. Left parietal lobe regions showed a greater response for multisyllabic as compared to monosyllabic words. This is consistent with clinical studies showing that left parietal lobe lesions can produce a deficit in speech programming. Despite similarities, covert and overt speech did not demonstrate the same patterns of BOLD response. The BOLD response was greater during overt speech in areas that have been shown to play an important role in speech production including left premotor cortex/BA6, left primary motor cortex, left insula, and left superior temporal gyrus.     

Homotopic language reorganization in the right hemisphere after early left hemisphere injury

Speech production is inextricably linked to speech perception, yet they are usually investigated in isolation. In this study, we employed a verbal-repetition task to identify the neural substrates of speech processing with two ends active simultaneously using functional MRI. Subjects verbally repeated auditory stimuli containing an ambiguous vowel sound that could be perceived as either a word or a pseudoword depending on the interpretation of the vowel. We found verbal repetition commonly activated the audition-articulation interface bilaterally at Sylvian fissures and superior temporal sulci. Contrasting word-versus-pseudoword trials revealed neural activities unique to word repetition in the left posterior middle temporal areas and activities unique to pseudoword repetition in the left inferior frontal gyrus. These findings imply that the tasks are carried out using different speech codes: an articulation-based code of pseudowords and an acoustic-phonetic code of words. It also supports the dual-stream model and imitative learning of vocabulary.     

Distributed processing and cortical specialization for speech and environmental sounds in human temporal cortex

Using functional MRI, we investigated whether auditory processing of both speech and meaningful non-linguistic environmental sounds in superior and middle temporal cortex relies on a complex and spatially distributed neural system. We found that evidence for spatially distributed processing of speech and environmental sounds in a substantial extent of temporal cortices. Most importantly, regions previously reported as selective for speech over environmental sounds also contained distributed information. The results indicate that temporal cortices supporting complex auditory processing, including regions previously described as speech-selective, are in fact highly heterogeneous.     

Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information

The functional specificity of different brain areas recruited in auditory language processing was investigated by means of event-related functional magnetic resonance imaging (fMRI) while subjects listened to speech input varying in the presence or absence of semantic and syntactic information. There were two sentence conditions containing syntactic structure, i.e., normal speech (consisting of function and content words), syntactic speech (consisting of function words and pseudowords), and two word-list conditions, i.e., real words and pseudowords. The processing of auditory language, in general, correlates with significant activation in the primary auditory cortices and in adjacent compartments of the superior temporal gyrus bilaterally. Processing of normal speech appeared to have a special status, as no frontal activation was observed in this case but was seen in the three other conditions. This difference may point toward a certain automaticity of the linguistic processes used during normal speech comprehension. When considering the three other conditions, we found that these were correlated with activation in both left and right frontal cortices. An increase of activation in the planum polare bilaterally and in the deep portion of the left frontal operculum was found exclusively when syntactic processes were in focus. Thus, the present data may be taken to suggest an involvement of the left frontal and bilateral temporal cortex when processing syntactic information during comprehension.     

Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information

The functional specificity of different brain regions recruited in auditory language processing was investigated by means of event-related functional magnetic resonance imaging (fMRI) while subjects listened to speech input varying in the presence or absence of semantic and syntactic information. There were two sentence conditions containing syntactic structure, i.e., normal speech (consisting of function and content words), syntactic speech (consisting of function words and pseudowords), and two word-list conditions, i.e., real words and pseudowords. The processing of auditory language, in general, correlates with significant activation in the primary auditory cortices and in adjacent compartments of the superior temporal gyrus bilaterally. Processing of normal speech appeared to have a special status, as no frontal activation was observed in this case but was seen in the other three conditions. This difference may point toward a certain automaticity of the linguistic processes used during normal speech comprehension. When considering the three other conditions, we found that these were correlated with activation in both left and right frontal cortices. An increase of activation in the planum polare bilaterally and in the deep portion of the left frontal operculum was found exclusively when syntactic processes were in focus. Thus, the present data may be taken to suggest an involvement of the left frontal and bilateral temporal cortex when processing syntactic information during comprehension.     

A mouse with a roof? effects of phonological neighbors on processing of words in sentences in a non-native language

The architecture of the language processing system for speakers of more than one language remains an intriguing topic of research. A common finding is that speakers of multiple languages are slower at responding to language stimuli in their non-native language (L2) than monolingual speakers. This may simply reflect participants' unfamiliarity with words in the L2, however it may also be the reflection of interference from competing lexical alternatives both across and within the participants' multiple languages. In the current studies (one behavioral, one electrophysiological) we investigate how interference from phonologically similar words within the L2 alone may account for problems in auditory language comprehension in non-native speakers. To this end a cross modal lexical priming (CMLP) paradigm was implemented, which allowed us to look for effects of spoken word primes embedded in sentences on the recognition of target stimuli. Specifically, we investigated whether a word such as mouse, which has a close phonological neighbor, house, would show a modulating effect on recognition of a word semantically related to house but not to mouse (i.e., roof). We hypothesized that L2 speakers, less efficient at categorizing phonemes in their L2 would show a difference in the processing of roof preceded by mouse as compared to roof preceded by another unrelated word, such as lamp, due to a residual co-activation of the phonological neighbor mouse. Furthermore, L1 speakers, highly proficient at recognizing phonemes in their native tongue, should show no such effect. The results of both studies clearly support our hypothesis, indicating that phonological neighbors in the L2 may greatly interfere with L2 word recognition.     

Disruption of spelling-to-sound correspondence mapping during single-word reading in patients with temporal lobe epilepsy

Processing and/or hemispheric differences in the neural bases of word recognition were examined in patients with long-standing, medically-intractable epilepsy localized to the left (N=18) or right (N=7) temporal lobe. Participants were asked to read words that varied in the frequency of their spelling-to-sound correspondences. For the right temporal lobe group, reaction times (RTs) showed the same pattern across spelling-to-sound correspondence conditions as previously reported for normal participants. For the left temporal lobe group, however, the pattern of RTs suggested a greater relative influence of orthographic frequency than rime frequency, such that performance was worse on words whose orthographic body was less frequent in the language. We discuss these results in terms of differences in processing between the two cerebral hemispheres: the results for the right-temporal lobe patients are taken to support connectionist models of reading as described for the dominant (left) hemisphere, while results for the left-temporal lobe patients support a view of the right hemisphere as relatively less sensitive to phonology and relatively more sensitive to orthography.     

Orthographic depth and the interaction of visual and auditory processing in word recognition

We studied the influence of word frequency and orthographic depth on the interaction of orthographic and phonetic information in word perception. Native speakers of English and Serbo-Croatian were presented with simultaneous printed and spoken verbal stimuli and had to decide whether they were equivalent. Decision reaction time was measured in three experimental conditions: Clear print and clear speech, degraded print and clear speech, and clear print and degraded speech. Within each language, the effects of visual and auditory degradation were measured, relative to the undegraded presentation. Both effects of degradation were much stronger in English than in Serbo-Croatian. Moreover, they were the same for high- and low-frequency words in both languages. These results can be accounted for by a parallel interactive processing model that assumes lateral connections between the orthographic and phonological systems at all of their levels. The structure of these lateral connections is independent of word frequency and is determined by the relationship between spelling and phonology in the language: simple isomorphic connections between graphemes and phonemes in Serbo-Croatian, but more complex, many-to-one, connections in English.     

汉语双字词在心理词典中的表征方式:来自fNIRS的证据

以汉语双字构成的真词与假词为实验材料,22名大学生为被试,采用功能性近红外脑成像技术(f NIRS)和事件相关设计,考察被试在完成词汇判断任务时的大脑激活模式,探索汉语双字词在心理词典中的表征方式。结果发现:(1)在完成真假词判断任务时,被试大脑左侧额叶和左侧颞叶均被激活;(2)与判断假词相比,被试在判断真词时显著地激活左额上回和左额中回。这一结果说明汉语双字词在心理词典中是混合表征的。     

All words are not created equal: expectations about word length guide infant statistical learning

Infants have been described as ‘statistical learners’ capable of extracting structure (such as words) from patterned input (such as language). Here, we investigated whether prior knowledge influences how infants track transitional probabilities in word segmentation tasks. Are infants biased by prior experience when engaging in sequential statistical learning? In a laboratory simulation of learning across time, we exposed 9- and 10-month-old infants to a list of either disyllabic or trisyllabic nonsense words, followed by a pause-free speech stream composed of a different set of disyllabic or trisyllabic nonsense words. Listening times revealed successful segmentation of words from fluent speech only when words were uniformly disyllabic or trisyllabic throughout both phases of the experiment. Hearing trisyllabic words during the pre-exposure phase derailed infants’ abilities to segment speech into disyllabic words, and vice versa. We conclude that prior knowledge about word length equips infants with perceptual expectations that facilitate efficient processing of subsequent language input.