Embodiment Effects and Language Comprehension in Alzheimer's Disease

It has been shown that when participants are asked to make sensibility judgments on sentences that describe a transfer of an object toward or away from their body, they are faster to respond when the response requires a movement in the same direction as the transfer described in the sentence. This phenomenon is known as the action compatibility effect (ACE). This study investigates whether the ACE exists for volunteers with Alzheimer's disease (AD), whether the ACE can facilitate language comprehension, and also whether the ACE can still be produced if the order of the two events is inverted, that is, whether overt movement can prime comprehension of transfer sentences. In Study 1, participants with AD, younger, and older adults were tested on an adaptation of the ACE Paradigm. In Study 2, the same paradigm was modified to include an arm movement that participants had to perform prior to sentence exposure on screen. In Study 1, young, older adults, and individuals with AD were faster to respond when the direction of the response movement matched the directionality implied by the sentence (ACE). In Study 2, no traditional ACE was found; participants were faster when the direction of the movement immediately preceding the sentence matched the directionality of the sentence. It was found that compatibility effects generated a relative advantage, that transfer schemata are easier to process, and that an ACE‐like effect can be the result of mutual priming between language and movement. Results suggested preservation in AD of the neural systems for action engaged during language comprehension, and conditions under which comprehension in AD can be facilitated in real life may be identified.     

Embodied Simulations Are Modulated by Sentential Perspective

There is considerable evidence that language comprehenders derive lexical‐semantic meaning by mentally simulating perceptual and motor attributes of described events. However, the nature of these simulations—including the level of detail that is incorporated and contexts under which simulations occur—is not well understood. Here, we examine the effects of first‐ versus third‐person perspective on mental simulations during sentence comprehension. First‐person sentences describing physical transfer towards or away from the body (e.g., “You threw the microphone,” “You caught the microphone”) modulated response latencies when responses were made along a front‐back axis, consistent with the action‐sentence compatibility effect (ACE). This effect was not observed for third‐person sentences (“He threw the microphone,” “He caught the microphone”). The ACE was observed when making responses along a left‐right axis for third‐person, but not first‐person sentences. Abstract sentences (e.g., “He heard the message”) did not show an ACE along either axis. These results show that perspective is a detail that is simulated during action sentence comprehension, and that motoric activations are flexible and affected by the pronominal perspective used in the sentence.     

Processing abstract language modulates motor system activity

Embodiment theory proposes that neural systems for perception and action are also engaged during language comprehension. Previous neuroimaging and neurophysiological studies have only been able to demonstrate modulation of action systems during comprehension of concrete language. We provide neurophysiological evidence for modulation of motor system activity during the comprehension of both concrete and abstract language. In Experiment 1, when the described direction of object transfer or information transfer (e.g., away from the reader to another) matched the literal direction of a hand movement used to make a response, speed of responding was faster than when the two directions mismatched (an action-sentence compatibility effect). In Experiment 2, we used single-pulse transcranial magnetic stimulation to study changes in the corticospinal motor pathways to hand muscles while reading the same sentences. Relative to sentences that do not describe transfer, there is greater modulation of activity in the hand muscles when reading sentences describing transfer of both concrete objects and abstract information. These findings are discussed in relation to the human mirror neuron system.     

The comprehension of action-related sentences may cause interference rather than facilitation on matching actions

This study used a dual-task paradigm to analyze the time course of motor resonance during the comprehension of action language. In the study, participants read sentences describing a transfer either away from (“I threw the tennis ball to my rival”) or toward themselves (“My rival threw me the tennis ball”). When the transfer verb appeared on the screen, and after a variable stimulus onset asynchrony (SOA), a visual motion cue (Experiment 1) or a static cue (Experiment 2) prompted participants to move their hand either away from or toward themselves to press a button. The results showed meaning–action interference at short SOAs and facilitation at the longest SOA for the matching conditions. These results support the hypothesis that motor processes associated with the comprehension of action-related language interfere with an overlapping motor task, whereas they facilitate a delayed motor task. These effects are discussed in terms of resonance processes in the motor cortex.     

Perception of motion affects language processing

Recently developed accounts of language comprehension propose that sentences are understood by constructing a perceptual simulation of the events being described. These simulations involve the re-activation of patterns of brain activation that were formed during the comprehender's interaction with the world. In two experiments we explored the specificity of the processing mechanisms required to construct simulations during language comprehension. Participants listened to (and made judgments on) sentences that described motion in a particular direction (e.g. "The car approached you"). They simultaneously viewed dynamic black-and-white stimuli that produced the perception of movement in the same direction as the action specified in the sentence (i.e. towards you) or in the opposite direction as the action specified in the sentence (i.e. away from you). Responses were faster to sentences presented concurrently with a visual stimulus depicting motion in the opposite direction as the action described in the sentence. This suggests that the processing mechanisms recruited to construct simulations during language comprehension are also used during visual perception, and that these mechanisms can be quite specific.     

Temporal dynamics of the action-sentence compatibility effect

A number of recent studies have demonstrated variants of the action-sentence compatibility effect (ACE), wherein the execution of a motor response is facilitated by the comprehension of sentences that describe actions taking place in the same direction as the motor response (e.g., a sentence about action towards one's body facilitates the execution of an arm movement towards the body). This paper presents an experiment that explores how the timing of the motor response during the processing of sentences affects the magnitude of the ACE that is observed. The results show that the ACE occurs when the motor response is executed at an early point in the comprehension of the sentence, disappears for a time, and then reappears when the motor response is executed right before the end of the sentence. These data help to refine our understanding of the temporal dynamics involved in the activation and use of motor information during sentence comprehension.     

The Action-Sentence Compatibility Effect: It's All in the Timing

When participants are asked to make sensibility judgments on sentences that describe action toward the body (i.e., "Mark dealt the cards to you") or away from the body (i.e., "You dealt the cards to Mark"), they are faster to respond when the response requires an arm movement in the same direction as the action described by the sentence. This congruence effect is known as the Action–Sentence Compatibility Effect (ACE). This study reports 4 experiments that extend our understanding of the ACE by exploring how the time at which one prepares the motor response required for the sensibility judgment affects the magnitude of the ACE. Results show that the ACE arises only when participants have the opportunity to plan their motor response while they are processing the sentence.     

Perception of auditory motion affects language processing

Previous reports have demonstrated that the comprehension of sentences describing motion in a particular direction (toward, away, up, or down) is affected by concurrently viewing a stimulus that depicts motion in the same or opposite direction. We report 3 experiments that extend our understanding of the relation between perception and language processing in 2 ways. First, whereas most previous studies of the relation between perception and language processing have focused on visual perception, our data show that sentence processing can be affected by the concurrent processing of auditory stimuli. Second, it is shown that the relation between the processing of auditory stimuli and the processing of sentences depends on whether the sentences are presented in the auditory or visual modality.     

高功能孤独症者非字面语义理解缺陷：隐喻视角的ERP研究

研究采用事件相关电位技术从隐喻语义理解角度考察了高功能孤独症成人非字面语义理解中的行为特点及脑半球参与模式。行为结果显示：高功能孤独症成人对新异隐喻句的反应时最长,两类隐喻句的反应时均长于普通成人,但错误率上两组被试间无差异。脑电结果显示：高功能孤独症组对新异隐喻句的N400波幅最大,未出现半球偏侧化现象,传统隐喻句在左半球的N400波幅大于右半球,两类隐喻句的N400波幅均大于正常被试。结论：高功能孤独症成人具备隐喻理解能力,并对新异隐喻的理解做出更多努力,但右脑功能异常仍旧存在,左右半球在总体功能或神经联通性上仍旧弱于常人。     

The action-sentence compatibility effect in Japanese sentences

The action-sentence compatibility effect (ACE) is a phenomenon in which a reader's response to a sentence is made faster when there is congruity between the action described in the sentence and the action that makes up the response. Previous studies showed the ACE occurs in action-related sentences in several languages. However, all these were SVO (verb-object) languages, in which verbs are placed before object nouns; this order is reversed in SOV languages. Moreover, those studies investigated hand responses. This study assessed the existence of the ACE in Japanese, an SOV language, and in foot responses. 24 female participants judged the sensibility of Japanese sentences that described actions and responded with either their foot or hand as an effector. Reaction times were significantly faster when there was congruity between the effector described in the sentences and the effector actually used for the response. However, sentence dependency was also found in the foot responses.     

Eye Movements Reveal Mental Looking Through Time

People often make use of a spatial “mental time line” to represent events in time. We investigated whether the eyes follow such a mental time line during online language comprehension of sentences that refer to the past, present, and future. Participants' eye movements were measured on a blank screen while they listened to these sentences. Saccade direction revealed that the future is mapped higher up in space than the past. Moreover, fewer saccades were made when two events are simultaneously taking place at the present moment compared to two events that are happening in different points in time. This is the first evidence that oculomotor correlates reflect mental looking along an abstract invisible time line during online language comprehension about time. Our results support the idea that observing eye movements is likely to “detect” invisible spatial scaffoldings which are involved in cognitively processing abstract meaning, even when the abstract meaning lacks an explicit spatial correlate. Theoretical implications of these findings are discussed.     

Sentence comprehension affects the dynamics of bimanual coordination: Implications for embodied cognition

Recent work in embodied cognition has demonstrated that language comprehension involves the motor system (e.g., Glenberg & Kaschak, 2002). Such findings are often attributed to mechanisms involving simulations of linguistically described events (Barsalou, 1999; Fischer & Zwaan, 2008). We propose that research paradigms in which simulation is the central focus need to be augmented with paradigms that probe the organization of the motor system during language comprehension. The use of well-studied motor tasks may be appropriate to this endeavour. To this end, we present a study in which participants perform a bimanual rhythmic task (Kugler & Turvey, 1987) while judging the plausibility of sentences. We show that the dynamics of the bimanual task differ when participants judge sentences describing performable actions as opposed to sentences describing events that are not performable. We discuss the general implications of our results for accounts of embodied cognition.     

The discovery of a miniature linguistic system: Function words and comprehension of an unfamiliar language

Two experiments investigated two complementary hypotheses: (a) The presence of semantically unimportant function words in sentences will not slow down the way subjects, unfamiliar with the language, extract the meaning of those sentences, provided suprasegmental cues are present, and conversely (b) the omission of such function words—as in native speakers' simplified speech to nonnative speakers—will not necessarily facilitate the task of meaning extraction, contrary to popular belief. In Experiment I, Dutch adult subjects were exposed to sentences in a miniature artificial language (MAL). In Experiment II, Greek adult subjects were exposed to sentences in an unfamiliar natural language (Dutch). In both experiments, the sentences contained eight words constituting a miniature linguistic system (mls), a system that subjects discovered through an audiovisual, concept-indentification task. The rate by which subjects performed this task was investigated as a function of two linguistic variables: (a) the presence or absence of semantically unimportant function words, and (b) the presence or absence of suprasegmental cues, in the input sentences, respectively. Results from both experiments confirmed the hypotheses (p<.001) The findings are discussed with reference to the linguistic versus cognitive simplicity issue, and to the comprehension versus production issue in language acquisition research. The methodological potential of the experimental paradigm for psycholinguistic research is also discussed.This paper forms part of a doctorate research project concerned with the comprehensibility of foreigner talk, and was supported by the Foundation for Linguistic Research, which is funded by the Netherlands Organization for Scientific Research, NWO.     

The misinterpretation of noncanonical sentences

Research on language comprehension has focused on the resolution of syntactic ambiguities, and most studies have employed garden-path sentences to determine the system's preferences and to assess its use of nonsyntactic sources information. A topic that has been neglected is how syntactically challenging but essentially unambiguous sentences are processed, including passives and object-clefts--sentences that require thematic roles to be assigned in an atypical order. The three experiments described here tested the idea that sentences are processed both algorithmically and heuristically. Sentences were presented aurally and the participants' task was to identify the thematic roles in the sentence (e.g., Who was the do-er?). The first experiment demonstrates that passives are frequently and systematically misinterpreted, especially when they express implausible ideas. The second shows that the surface frequency of a syntactic form does not determine ease of processing, as active sentences and subject-clefts were comprehended equally easily despite the rareness of the latter type. The third experiment compares the processing of subject- and object-clefts, and the results show that they are similar to actives and passives, respectively, again despite the infrequent occurrence in English of any type of cleft. The results of the three experiments suggest that a comprehensive theory of language comprehension must assume that simple processing heuristics are used during processing in addition to (and perhaps sometimes instead of) syntactic algorithms. Moreover, the experiments support the idea that language processing is often based on shallow processing, yielding a merely "good enough" rather than a detailed linguistic representation of an utterance's meaning.     

Language comprehenders mentally represent the shapes of objects

We examined the prediction that people activate perceptual symbols during language comprehension. Subjects read sentences describing an animal or object in a certain location. The shape of the object or animal changed as a function of its location (e.g., eagle in the sky, eagle in a nest). However, this change was only implied by the sentences. After reading a sentence, subjects were presented with a line drawing of the object in question. They judged whether the object had been mentioned in the sentence (Experiment 1) or simply named the object (Experiment 2). In both cases, responses were faster when the pictured object's shape matched the shape implied by the sentence than when there was a mismatch. These results support the hypothesis that perceptual symbols are routinely activated in language comprehension.     

Musical metaphors: Evidence for a spatial grounding of non-literal sentences describing auditory events

This study investigated whether the spatial terms high and low, when used in sentence contexts implying a non-literal interpretation, trigger similar spatial associations as would have been expected from the literal meaning of the words. In three experiments, participants read sentences describing either a high or a low auditory event (e.g., The soprano sings a high aria vs. The pianist plays a low note). In all Experiments, participants were asked to judge (yes/no) whether the sentences were meaningful by means of up/down (Experiments 1 and 2) or left/right (Experiment 3) key press responses. Contrary to previous studies reporting that metaphorical language understanding differs from literal language understanding with regard to simulation effects, the results show compatibility effects between sentence implied pitch height and response location. The results are in line with grounded models of language comprehension proposing that sensory motor experiences are being elicited when processing literal as well as non-literal sentences.     

Induction of augmented transition networks

LAS is a program that acquires augmented transition network (ATN) grammars. It requires as data sentences of the language and semantic network representatives of their meaning. In acquiring the ATN grammars, it induces the word classes of the language, the rules of formation for sentences, and the rules mapping sentences onto meaning. The induced ATN grammar can be used both for sentence generation and sentence comprehension. Critical to the performance of the program are assumptions that it makes about the relation between sentence structure and surface structure (the graph deformation condition), about when word classes may be formed and when ATN networks may be merged, and about the structure of noun phrases. These assumptions seem to be good heuristics which are largely true for natural languages although they would not be true for many nonnatural languages. Provided these assumptions are satisfied LAS seems capable of learning any context-free language.     

动词理解中空间表征的激活过程

探讨动词理解中空间表征的激活过程。被试听以不同空间轴向动词作谓语的句子后辨认视觉刺激的形状。实验1以肯定句为材料探讨语义理解对视知觉任务的影响。实验2以客观原因否定句为材料排除句子表征的影响。实验3以主观原因否定句为材料探讨动词空间元素的激活机制。总的结果表明,动词理解过程中激活其表征中的空间元素,这种激活是自动的、非策略性的,不受情境中客观原因或主观意愿否定的影响。动词理解空间效应反映了语言表征中的知觉运动特征     

Memory-load interference in syntactic processing

Participants remembered a short set of words while reading syntactically complex sentences (object–extracted clefts) and syntactically simpler sentences (subject–extracted clefts) in a memory–load study. The study also manipulated whether the words in the set and the words in the sentence were of matched or unmatched types (common nouns vs. proper names). Performance in sentence comprehension was worse for complex sentences than for simpler sentences, and this effect was greater when the type of words in the memory load matched the type of words in the sentence. These results indicate that syntactic processing is not modular, instead suggesting that it relies on working memory resources that are used for other nonsyntactic processes. Further, the results indicate that similarity–based interference is an important constraint on information processing that can be overcome to some degree during language comprehension by using the coherence of language to construct integrated representations of meaning.