Dissociation and association of the embodied representation of tool-use verbs and hand verbs: An fMRI study

Embodied semantic theories suppose that representation of word meaning and actual sensory-motor processing are implemented in overlapping systems. According to this view, association and dissociation of different word meaning should correspond to dissociation and association of the described sensory-motor processing. Previous studies demonstrate that although tool-use actions and hand actions have overlapping neural substrates, tool-use actions show greater activations in frontal–parietal–temporal regions that are responsible for motor control and tool knowledge processing. In the present study, we examined the association and the dissociation of the semantic representation of tool-use verbs and hand action verbs. Chinese verbs describing tool-use or hand actions without tools were included, and a passive reading task was employed. All verb conditions showed common activations in areas of left middle frontal gyrus, left inferior frontal gyrus (BA 44/45) and left inferior parietal lobule relative to rest, and all conditions showed significant effects in premotor areas within the mask of hand motion effects. Contrasts between tool-use verbs and hand verbs demonstrated that tool verbs elicited stronger activity in left superior parietal lobule, left middle frontal gyrus and left posterior middle temporal gyrus. Additionally, psychophysiological interaction analyses demonstrated that tool verbs indicated greater connectivity among these regions. These results suggest that the brain regions involved in tool-use action processing also play more important roles in tool-use verb processing and that similar systems may be responsible for word meaning representation and actual sensory-motor processing.     

Do Children Understand Iconic Gestures About Events as Early as Iconic Gestures About Entities?

Children can understand iconic co-speech gestures that characterize entities by age 3 (Stanfield et al. in J Child Lang 40(2):1–10, 2014; e.g., “I’m drinking” \(+\) tilting hand in C-shape to mouth as if holding a glass). In this study, we ask whether children understand co-speech gestures that characterize events as early as they do so for entities, and if so, whether their understanding is influenced by the patterns of gesture production in their native language. We examined this question by studying native English speaking 3- to 4 year-old children and adults as they completed an iconic co-speech gesture comprehension task involving motion events across two studies. Our results showed that children understood iconic co-speech gestures about events at age 4, marking comprehension of gestures about events one year later than gestures about entities. Our findings also showed that native gesture production patterns influenced children’s comprehension of gestures characterizing such events, with better comprehension for gestures that follow language-specific patterns compared to the ones that do not follow such patterns—particularly for manner of motion. Overall, these results highlight early emerging abilities in gesture comprehension about motion events.     

Effective brain connectivity in children with reading difficulties during phonological processing

Using Dynamic Causal Modeling (DCM) and functional magnetic resonance imaging (fMRI), we examined effective connectivity between three left hemisphere brain regions (inferior frontal gyrus, inferior parietal lobule, fusiform gyrus) and bilateral medial frontal gyrus in 12 children with reading difficulties (M age=12.4, range: 8.11-14.10) and 12 control children (M age=12.3, range: 8.9-14.11) during rhyming judgments to visually presented words. More difficult conflicting trials either had similar orthography but different phonology (e.g. pint-mint) or similar phonology but different orthography (e.g. jazz-has). Easier non-conflicting trials had similar orthography and phonology (e.g. dime-lime) or different orthography and phonology (e.g. staff-gain). The modulatory effect from left fusiform gyrus to left inferior parietal lobule was stronger in controls than in children with reading difficulties only for conflicting trials. Modulatory effects from left fusiform gyrus and left inferior parietal lobule to left inferior frontal gyrus were stronger for conflicting trials than for non-conflicting trials only in control children but not in children with reading difficulties. Modulatory effects from left inferior frontal gyrus to inferior parietal lobule, from medial frontal gyrus to left inferior parietal lobule, and from left inferior parietal lobule to medial frontal gyrus were positively correlated with reading skill only in control children. These findings suggest that children with reading difficulties have deficits in integrating orthography and phonology utilizing left inferior parietal lobule, and in engaging phonological rehearsal/segmentation utilizing left inferior frontal gyrus possibly through the indirect pathway connecting posterior to anterior language processing regions, especially when the orthographic and phonological information is conflicting.     

Functional neuroanatomy of gesture–speech integration in children varies with individual differences in gesture processing

Gesture is an integral part of children's communicative repertoire. However, little is known about the neurobiology of speech and gesture integration in the developing brain. We investigated how 8‐ to 10‐year‐old children processed gesture that was essential to understanding a set of narratives. We asked whether the functional neuroanatomy of gesture–speech integration varies as a function of (1) the content of speech, and/or (2) individual differences in how gesture is processed. When gestures provided missing information not present in the speech (i.e., disambiguating gesture; e.g., “pet” + flapping palms = bird), the presence of gesture led to increased activity in inferior frontal gyri, the right middle temporal gyrus, and the left superior temporal gyrus, compared to when gesture provided redundant information (i.e., reinforcing gesture; e.g., “bird” + flapping palms = bird). This pattern of activation was found only in children who were able to successfully integrate gesture and speech behaviorally, as indicated by their performance on post‐test story comprehension questions. Children who did not glean meaning from gesture did not show differential activation across the two conditions. Our results suggest that the brain activation pattern for gesture–speech integration in children overlaps with—but is broader than—the pattern in adults performing the same task. Overall, our results provide a possible neurobiological mechanism that could underlie children's increasing ability to integrate gesture and speech over childhood, and account for individual differences in that integration.     

Neural responses to meaningless pseudosigns: evidence for sign-based phonetic processing in superior temporal cortex

To identify neural regions that automatically respond to linguistically structured, but meaningless manual gestures, 14 deaf native users of American Sign Language (ASL) and 14 hearing non-signers passively viewed pseudosigns (possible but non-existent ASL signs) and non-iconic ASL signs, in addition to a fixation baseline. For the contrast between pseudosigns and baseline, greater activation was observed in left posterior superior temporal sulcus (STS), but not in left inferior frontal gyrus (BA 44/45), for deaf signers compared to hearing non-signers, based on VOI analyses. We hypothesize that left STS is more engaged for signers because this region becomes tuned to human body movements that conform the phonological constraints of sign language. For deaf signers, the contrast between pseudosigns and known ASL signs revealed increased activation for pseudosigns in left posterior superior temporal gyrus (STG) and in left inferior frontal cortex, but no regions were found to be more engaged for known signs than for pseudosigns. This contrast revealed no significant differences in activation for hearing non-signers. We hypothesize that left STG is involved in recognizing linguistic phonetic units within a dynamic visual or auditory signal, such that less familiar structural combinations produce increased neural activation in this region for both pseudosigns and pseudowords.     

动词论元结构复杂性加工的认知神经机制

动词论元结构复杂性表现在论元数量、论元范畴选择模式、题元角色指派模式和映射方式四个方面。大部分实证研究表明, 更多的论元数量、选择性论元范畴、选择性题元角色指派以及非典型映射, 使动词论元结构加工的认知神经机制更复杂。多论元加工功能脑区主要涉及左侧额下回和外侧裂周后部; 选择性论元范畴加工功能脑区主要涉及左侧额下回、额叶中后部、颞上回和颞叶中后部; 选择性题元角色指派加工功能脑区主要涉及外侧裂周后部、左侧额叶中后部和额下回; 非典型映射加工功能脑区主要涉及左侧额下回、颞上回、颞中回和颞叶后部。左侧额下回可能涉及初始句法加工、动词次范畴确定、句法移位和非宾格动词语义加工, 左侧额叶中后部可能涉及初始句法加工和动词次范畴确定, 左侧颞上回和颞叶中后部可能涉及表层句法加工和表层论元句法-语义整合, 外侧裂周后部可能涉及论元语义表征。动词论元结构加工过程和动词词汇特征表明, 复杂性某些方面存在交互作用。动词论元结构复杂性与加工难易的对应关系、复杂性加工难度层级和交互作用的认知神经机制以及汉语动词论元结构复杂性加工认知神经机制等议题, 有待进一步探讨。     

抑郁障碍和焦虑障碍治疗的神经心理机制——脑成像研究的ALE元分析

抑郁障碍和焦虑障碍是两种最常见的精神障碍。使用激活似然估计(activation likelihood estimation,ALE)元分析对抑郁和焦虑障碍患者治疗后出现的一致脑区激活改变进行评估,并考察不同条件下这一改变的差异。研究共纳入25篇文献,结果发现:(1)抑郁和焦虑障碍接受治疗后,枕下回(inferior occipital gyrus,IOG)等脑区激活增加;豆状核(lentiform nucleus)等激活减少。(2)a.心理治疗产生的脑激活改变为豆状核活动的减少;药物治疗则在于扣带回(cingulate gyrus)等活动增加,而楔前叶(precuneus)等活动减少。b.任务下成像治疗后激活增加的脑区为扣带回等,减少的脑区为楔前叶等;静息下成像,治疗后枕下回等激活增加,额内侧回(medial frontal gyrus,MFG)等激活减少。c.抑郁障碍治疗后的脑激活变化为扣带回等活动增加,楔前叶等活动减少;焦虑障碍在于前扣带回/额内侧回(anterior cingulate/MFG)活动减少。研究表明治疗会给抑郁和焦虑障碍带来一致的脑区激活改变;治疗方法、成像状态和障碍类型不同,治疗后脑区激活改变也存在差异。     

Parietal versus temporal lobe components in spatial cognition: Setting the mid‐point of a horizontal line

Recent anatomo‐clinical correlation studies have extended to the superior temporal gyrus, the right hemisphere lesion sites associated with the left unilateral spatial neglect, in addition to the traditional posterior‐inferior‐parietal localization of the responsible lesion (supramarginal gyrus, at the temporo‐parietal junction). The study aimed at teasing apart, by means of repetitive transcranial magnetic stimulation (rTMS), the contribution of the inferior parietal lobule (angular gyrus versus supramarginal gyrus) and of the superior temporal gyrus of the right hemisphere, in making judgments about the mid‐point of a horizontal line, a widely used task for detecting and investigating spatial neglect. rTMS trains at 25 Hz frequency were delivered over the inferior parietal lobule (angular gyrus and supramarginal gyrus), the superior temporal gyrus and the anterior parietal lobe of the right hemisphere, in 10 neurologically unimpaired participants, performing a line bisection judgment task. rTMS of the inferior parietal lobule at the level of the supramarginal gyrus brought about a rightward error in the bisection judgment, ipsilateral to the side of the rTMS, with stimulation over the other sites being ineffective. The neural correlates of computing the mid‐point of a horizontal segment include the right supramarginal gyrus in the inferior parietal lobule and do not extend to the angular gyrus and the superior temporal gyrus. These rTMS data in unimpaired subjects constrain the evidence from lesion studies in brain‐damaged patients, emphasizing the major role of a subset of relevant regions.     

Motor-iconicity of sign language does not alter the neural systems underlying tool and action naming

Positron emission tomography was used to investigate whether the motor-iconic basis of certain forms in American Sign Language (ASL) partially alters the neural systems engaged during lexical retrieval. Most ASL nouns denoting tools and ASL verbs referring to tool-based actions are produced with a handshape representing the human hand holding a tool and with an iconic movement depicting canonical tool use, whereas the visual iconicity of animal signs is more idiosyncratic and inconsistent across signs. We investigated whether the motor-iconic relation between a sign and its referent alters the neural substrate for lexical retrieval in ASL. Ten deaf native ASL signers viewed photographs of tools/utensils or of actions performed with or without an implement and were asked to overtly produce the ASL sign for each object or action. The control task required subjects to judge the orientation of unknown faces. Compared to the control task, naming tools engaged left inferior and middle frontal gyri, bilateral parietal lobe, and posterior inferotemporal cortex. Naming actions performed with or without a tool engaged left inferior frontal gyrus, bilateral parietal lobe, and posterior middle temporal gyrus at the temporo-occipital junction (area MT). When motor-iconic verbs were compared with non-iconic verbs, no differences in neural activation were found. Overall, the results indicate that even when the form of a sign is indistinguishable from a pantomimic gesture, the neural systems underlying its production mirror those engaged when hearing speakers name tools or tool-based actions with speech.     

The homophone effect during visual word recognition in children: an fMRI study

Functional magnetic resonance imaging was used to investigate the role of phonology in visual word recognition (VWR). A group of children between the ages of 7 and 13 participated in a lexical decision task in which lexical frequency and homophony were manipulated. A significant homophone effect was observed for the high-frequency condition, indicating that phonology does indeed play a significant role in VWR. The brain activation patterns also support this idea in that regions that have been linked to phonological processing, the inferior frontal gyrus and the inferior parietal lobe, also revealed a homophone effect. Additionally, the posterior superior temporal cortex showed a homophone effect; however, this activation is argued to be related to lexical competition generated by the high-frequency homophone via the activation of multiple semantic representations.     

时空隐喻的心理现实性:手势和手语的视角

早期有关时空隐喻表征心理现实性的研究因囿于有声语言而饱受批评,近年来兴起的手势和手语研究为该问题的证明提供了新的视角和证据。一方面,手势在三个空间维度上均可形成时空隐喻,有力地回应了反对派对概念隐喻理论在语言和概念层面循环论证的质疑;另一方面,手语空间运用的独特性以及文化图式的复杂性,导致其时空隐喻表现形式更为多样,为该领域研究提供了更加丰富的类型学证据。未来研究还应当注意心理学、语言学以及民族学等多学科的交汇融合,建立起概括力更强和系统性更完整的理论框架,将口语、手势和手语同时囊括其中。     

Expert athletes activate somatosensory and motor planning regions of the brain when passively listening to familiar sports sounds

The present functional magnetic resonance imaging study examined the neural response to familiar and unfamiliar, sport and non-sport environmental sounds in expert and novice athletes. Results revealed differential neural responses dependent on sports expertise. Experts had greater neural activation than novices in focal sensorimotor areas such as the supplementary motor area, and pre- and postcentral gyri. Novices showed greater activation than experts in widespread areas involved in perception (i.e. supramarginal, middle occipital, and calcarine gyri; precuneus; inferior and superior parietal lobules), and motor planning and processing (i.e. inferior frontal, middle frontal, and middle temporal gyri). These between-group neural differences also appeared as an expertise effect within specific conditions. Experts showed greater activation than novices during the sport familiar condition in regions responsible for auditory and motor planning, including the inferior frontal gyrus and the parietal operculum. Novices only showed greater activation than experts in the supramarginal gyrus and pons during the non-sport unfamiliar condition, and in the middle frontal gyrus during the sport unfamiliar condition. These results are consistent with the view that expert athletes are attuned to only the most familiar, highly relevant sounds and tune out unfamiliar, irrelevant sounds. Furthermore, these findings that athletes show activation in areas known to be involved in action planning when passively listening to sounds suggests that auditory perception of action can lead to the re-instantiation of neural areas involved in producing these actions, especially if someone has expertise performing the actions.     

Action observation network in childhood: a comparative fMRI study with adults

Very little is known about the action observation network and the mirror neuron system (AON/MNS) in children and its age‐related properties compared with those observed in adults. In the present fMRI study we explored the activation of areas belonging to the AON/MNS in children and adults during observation of complex hand‐grasping actions, as compared to observation of simple grasping acts executed with the left and the right hand, seen from a first person perspective. The results indicate that during the action observation tasks in children there was activation of a cortical network similar to that found in adults, including the premotor cortex, the posterior part of the inferior frontal gyrus and the posterior parietal lobe. However, the activation in children was more widespread and showed a higher inter‐subject variability compared with adults. Furthermore, the activated network seems more lateralized to the left hemisphere in adults and more bilateral in children, with a linear growth of lateralization index as a function of age. Finally, in children the activation in the anterior intraparietal cortex (AIP) of each hemisphere was higher during observation of the contralateral hand (hand identity effect) and during the observation of complex actions relative to simple grasping acts, confirming the role of AIP for action‐related hand identity previously described in adults. These results support the assumption that structure and size of action representations are sensitive to mechanisms of development and show physiological plasticity. These properties of the AON/MNS could constitute a powerful tool for spontaneous reorganization and recovery of motor deficits after brain injury in children and in adults, as well as for specific rehabilitation programmes.     

Large,colorful, or noisy? Attribute- and modality-specific activations during retrieval of perceptual attribute knowledge

Position emission tomography was used to investigate whether retrieval of perceptual knowledge from long-term memory activates unique cortical regions associated with the modality and/or attribute type retrieved. Knowledge about the typical color, size, and sound of common objects and animals was probed, in response to written words naming the objects. Relative to a nonsemantic control task, all the attribute judgments activated similar left temporal and frontal regions. Visual (color, size) knowledge selectively activated the right posterior inferior temporal (PIT) cortex, whereas sound judgments elicited selective activation in the left posterior superior temporal gyrus and the adjacent parietal cortex. All of the attribute judgments activated a left PIT region, but color retrieval generated more activation in this area. Size judgments activated the right medial parietal cortex. These results indicate that the retrieval of perceptual semantic information activates not only a general semantic network, but also cortical areas specialized for the modality and attribute type of the knowledge retrieved.     

Neuropsychological functions of hand movements and gestures change in the presence or absence of speech

In order to address previous controversies whether hand movements and gestures are linked to mental concepts or solely to the process of speaking, in the present study we investigate the neuropsychological functions of the entire spectrum of unimanual and bimanual hand movements and gestures when they either accompany speaking or when they act as the only means to communicate in the absence of speech. The results showed that the hand movement activity regarding all types of hand movements and gestures stayed constant with and without speaking. The analysis of the Structure of hand movements showed that executions shifted from in space hand movements with a phase structure during the condition without speech to more irregular on body hand movements without a phase structure during the co-speech condition. The gestural analysis revealed that pantomime gestures increase under conditions without speech whereas emotional motions and subject-oriented actions primarily occur when speaking. The present results provide evidence that the overall hand movement activity does not differ between co-speech conditions and conditions without speech, but that the hands adopt different neuropsychological functions. We conclude that the hands primarily externalise mental concepts in conditions without speaking but that their use shifts to more self-regulation and to endorsing verbal output with emotional connotations when they accompany speech.     

左侧额下回在句子语义整合加工中的作用

语义整合帮助人们在阅读理解中将小信息块整合成一个完整、连贯的句子意义表达, 是阅读理解中非常重要的认知过程。通过对比40多篇句子加工相关的脑机制研究, 发现左侧额下回在fMRI研究中是参与语义整合加工激活概率最高的区域, 而颞叶及后部是MEG研究中激活概率最高的区域。另外, 左侧额下回是如何参与语义整合、它在内隐和外显语义整合中的机制是否相同、以及这种整合加工与一般的控制性加工、词汇启动的关系都是研究者广泛关注的问题。本文对上述问题进行了详细的综述和讨论。     

Listening to polyphonic music recruits domain-general attention and working memory circuits

Polyphonic music combines multiple auditory streams to create complex auditory scenes, thus providing a tool for investigating the neural mechanisms that orient attention in natural auditory contexts. Across two fMRI experiments, we varied stimuli and task demands in order to identify the cortical areas that are activated during attentive listening to real music. In individual experiments and in a conjunction analysis of the two experiments, we found bilateral blood oxygen level dependent (BOLD) signal increases in temporal (the superior temporal gyrus), parietal (the intraparietal sulcus), and frontal (the precentral sulcus, the inferior frontal sulcus and gyrus, and the frontal operculum) areas during selective and global listening, as compared with passive rest without musical stimulation. Direct comparisons of the listening conditions showed significant differences between attending to single timbres (instruments) and attending across multiple instruments, although the patterns that were observed depended on the relative demands of the tasks being compared. The overall pattern of BOLD signal increases indicated that attentive listening to music recruits neural circuits underlying multiple forms of working memory, attention, semantic processing, target detection, and motor imagery. Thus, attentive listening to music appears to be enabled by areas that serve general functions, rather than by music-specific cortical modules.     

Why the spontaneous images created by the hands during talk can help make TV advertisements more effective

The design of effective communications depends upon an adequate model of the communication process. The traditional model is that speech conveys semantic information and bodily movement conveys information about emotion and interpersonal attitudes. But McNeill (2000) argues that this model is fundamentally wrong and that some bodily movements, namely spontaneous hand movements generated during talk (iconic gestures), are integral to semantic communication. But can we increase the effectiveness of communication using this new theory? Focusing on advertising we found that advertisements in which the message was split between speech and iconic gesture (possible on TV) were significantly more effective than advertisements in which meaning resided purely in speech or language (radio/ newspaper). We also found that the significant differences in communicative effectiveness were maintained across five consecutive trials. We compared the communicative power of professionally made TV advertisements in which a spoken message was accompanied either by iconic gestures or by pictorial images, and found the iconic gestures to be more effective. We hypothesized that iconic gestures are so effective because they illustrate and isolate just the core semantic properties of a product. This research suggests that TV advertisements can be made more effective by incorporating iconic gestures with exactly the right temporal and semantic properties.     

Neural correlates of bimodal speech and gesture comprehension

The present study examined the neural correlates of speech and hand gesture comprehension in a naturalistic context. Fifteen participants watched audiovisual segments of speech and gesture while event-related potentials (ERPs) were recorded to the speech. Gesture influenced the ERPs to the speech. Specifically, there was a right-lateralized N400 effect-reflecting semantic integration-when gestures mismatched versus matched the speech. In addition, early sensory components in bilateral occipital and frontal sites differentiated speech accompanied by matching versus non-matching gestures. These results suggest that hand gestures may be integrated with speech at early and late stages of language processing.     

Neuroanatomical correlates of oral reading in acute left hemispheric stroke

Oral reading is a complex skill involving the interaction of orthographic, phonological, and semantic processes. Functional imaging studies with nonimpaired adult readers have identified a widely distributed network of frontal, inferior parietal, posterior temporal, and occipital brain regions involved in the task. However, while functional imaging can identify cortical regions engaged in the process under examination, it cannot identify those brain regions essential for the task. The current study aimed to identify those neuroanatomical regions critical for successful oral reading by examining the relationship between word and nonword oral reading deficits and areas of tissue dysfunction in acute stroke. We evaluated 91 patients with left hemisphere ischemic stroke with a test of oral word and nonword reading, and magnetic resonance diffusion-weighted and perfusion-weighted imaging, within 24-48 h of stroke onset. A voxel-wise statistical map showed that impairments in word and nonword reading were associated with a distributed network of brain regions, including the inferior and middle frontal gyri, the middle temporal gyrus, the supramarginal and angular gyri, and the middle occipital gyrus. In addition, lesions associated with word deficits were found to be distributed more frontally, while nonword deficits were associated with lesions distributed more posteriorly.