Verb aspect and perceptual simulations

Two experiments investigate the influence of verb aspect on situation representations. The results demonstrate that comprehenders use verb aspect as a cue to regulate the activation of ongoing simulations of situations over time. Experiment 1 measured word-by-word reading as well as sensibility judgements on sentences in which a target object word had been replaced by a picture. For the past imperfective sentences, participants were faster to process the picture, the two words following the picture, and the sensibility judgements when objects were pictured in use rather than not in use. However, this in-use facilitation was limited to processing of the picture for the past perfect sentences. Experiment 2 served as a control to ensure that the use effect and its interaction with verb aspect were a result of contextual manipulations rather than surface features of the pictures themselves. The results are interpreted within the framework of perceptual simulations during language comprehension.     

Is Complex Visual Information Implicated During Language Comprehension? The Case of Cast Shadows

Previous research showed that sensorimotor information affects the perception of properties associated with implied perceptual context during language comprehension. Three experiments addressed a novel question of whether perceptual context may contribute to a simulation of information about such out-of-sight objects as cast shadows. In Experiment 1, participants read a sentence that implied a particular shadow cast on a target (blinds vs. an open window) and then verified the picture of the object onto which a shadow was cast. Responses were faster when the shadow of blinds cast on the object matched that implied by the sentence. However, the data did not show the same matching effect for pictures with cast shadows from an open window. In Experiments 2 and 3, we found that verification times for pictures with no cast shadows were faster when preceded by an “open window” sentence, thus suggesting that reading the sentence does not elicit a visual simulation of any specific shadow. Experiment 3 showed that the objects superimposed with a cast shadow of the blinds and blinds themselves were verified faster after reading a “blinds” sentence. However, the results of an order analysis showed the temporal stability of the “blinds shadows” effect, but the disappearance of the “blinds” effect in the second half of the data. We conclude that the results are compatible, to a lesser or greater extent, with multiple accounts, and discuss our findings in the context of a mental imagery view, a mental simulation view, and an amodal representation view.     

How does verb aspect constrain event representations?

We investigated the relative contribution of perfective and imperfective aspectual cues on situation models. In Experiment 1, participants were more likely to choose pictures showing completed events than pictures showing ongoing events when they had read perfective sentences, but chose either picture after reading imperfective sentences. In Experiment 2, only one picture was presented and participants were faster to respond to completed pictures than to ongoing pictures when they had read perfective sentences, but showed no latency differences after reading the imperfective sentences. In Experiment 3, participants were faster to read perfective sentences after having seen completed pictures rather than intermediate pictures, but there was no difference for imperfective sentences. The consistent pattern of results demonstrates that readers construct mental representations of completed events when the perfective aspect is used to describe an event. The lack of effect on imperfective sentences and pictures suggests that each reader represents an in-progress event at varying stages of completion.     

汉语句子理解中物体隐含颜色心理模拟的ERPs研究

当前研究采用事件相关电位(ERPs)技术,同时沿用经典的句图匹配范式,考察句子理解中物体典型颜色与非典型颜色的心理加工过程在脑电活动上的反映。实验中被试先阅读句子再判断句子后呈现的图片物体是否在句子中出现过,句子中隐含的物体颜色或是关键物体的典型颜色或是非典型颜色。实验结果发现,典型颜色句子版本下句图不匹配条件比句图匹配条件引发了更大的N400效应;而非典型颜色句子版本下两者N400差异不显著。研究结果表明,人们在汉语句子理解过程中会实时对隐含的物体颜色信息进行心理模拟。并且,句子隐含物体颜色的典型性是造成匹配易化或不匹配易化的关键因素之一。     

Language and age effects in children's processing of word order

We compare the processing of transitive sentences in young learners of a strict word order language (English) and two languages that allow noun omissions and many variant word orders: Turkish, a case-marked language, and Mandarin Chinese, a non case-marked language. Children aged 1–3 years listened to simple transitive sentences in the typical word order of their language, paired with two visual scenes, only one of which matched the sentence. Multiple measures of comprehension (percent of looking to match, latency to look to match, number of switches of attention) revealed a general pattern of early sensitivity to word order, coupled with language and age effects in children's processing efficiency. In particular, English learners showed temporally speedier processing of transitive sentences than Turkish learners, who also displayed more uncertainty about the matching scene. Mandarin learners behaved like Turkish learners in showing slower processing of sentences, and all language groups displayed faster processing by older than younger children. These results demonstrate that sentence processing is sensitive to crosslinguistic features beginning early in language development.     

Integration of perceptual information in word access

According to experiential theories of language comprehension, perceptual information plays an essential role when word meanings are accessed. We conducted four experiments to investigate how different types of perceptual information such as colour and shape are combined during word access. One possibility is that the colour and shape of a word's referent are activated independently from one another and are combined in an additive manner. Alternatively, words might activate perceptual representations via a multiplicative integration of colour and shape. Experiment 1 established that participants follow a multiplicative similarity rule when they judge the similarity of schematic pictures to actual fruits and vegetables. In Experiments 2 to 4, participants performed a classification task, a lexical decision task, or a word-naming task on names of fruits and vegetables that were superimposed on a background picture. Responses were facilitated only when both colour and shape of the picture matched the word's referents. Response times were associated negatively with mean similarity ratings and the consistency of these ratings obtained in the first experiment. These results suggest a multiplicative integration of different types of perceptual information during word access.     

Moving words: dynamic representations in language comprehension

Eighty-two participants listened to sentences and then judged whether two sequentially presented visual objects were the same. On critical trials, participants heard a sentence describe the motion of a ball toward or away from the observer (e.g., “The pitcher hurled the softball to you”). Seven hundred and fifty milliseconds after the offset of the sentence, a picture of an object was presented for 500 ms, followed by another picture. On critical trials, the two pictures depicted the kind of ball mentioned in the sentence. The second picture was displayed 175 ms after the first. Crucially, it was either slightly larger or smaller than the first picture, thus suggesting movement of the ball toward or away from the observer. Participants responded more quickly when the implied movement of the balls matched the movement described in the sentence. This result provides support for the view that language comprehension involves dynamic perceptual simulations.     

语境与颜色诊断性对句子理解中物体隐含颜色知觉模拟的影响

具身认知理论的代表知觉符号系统认为知识表征具有知觉特点,并假设语言理解中对于所涉及概念及其隐含属性会即时进行知觉模拟。当前研究采用句图匹配范式,通过三个实验探讨句子理解中是否会对物体隐含颜色属性进行知觉模拟及其具体条件。实验1沿用经典范式验证语言理解中对物体隐含颜色存在知觉模拟过程。实验2通过变化句子前语境考察语境是否支持对于物体隐含颜色知觉模拟的影响。实验3进一步考察当颜色属性与其对应物体存在稳定关联时(即具有高颜色诊断性的物体),在不同语境作用下是否会引起知觉表征的变化。研究结果表明,语言理解中知觉模拟过程是灵活的,语境改变会引起读者所激活的物体颜色知觉表征发生变化,并且对于高颜色诊断物体,即使缺乏支持语境,读者仍然会对句中隐含的物体特定颜色进行知觉模拟。     

Encouraging prediction during production facilitates subsequent comprehension: Evidence from interleaved object naming in sentence context and sentence reading

Many studies have shown that a supportive context facilitates language comprehension. A currently influential view is that language production may support prediction in language comprehension. Experimental evidence for this, however, is relatively sparse. Here we explored whether encouraging prediction in a language production task encourages the use of predictive contexts in an interleaved comprehension task. In Experiment 1a, participants listened to the first part of a sentence and provided the final word by naming aloud a picture. The picture name was predictable or not predictable from the sentence context. Pictures were named faster when they could be predicted than when this was not the case. In Experiment 1b the same sentences, augmented by a final spill-over region, were presented in a self-paced reading task. No difference in reading times for predictive versus non-predictive sentences was found. In Experiment 2, reading and naming trials were intermixed. In the naming task, the advantage for predictable picture names was replicated. More importantly, now reading times for the spill-over region were considerable faster for predictive than for non-predictive sentences. We conjecture that these findings fit best with the notion that prediction in the service of language production encourages the use of predictive contexts in comprehension. Further research is required to identify the exact mechanisms by which production exerts its influence on comprehension.     

Children's comprehension of sentences with focus particles

We report three studies investigating children's and adults' comprehension of sentences containing the focus particle only. In Experiments 1 and 2, four groups of participants (6-7 years, 8-10 years, 11-12 years and adult) compared sentences with only in different syntactic positions against pictures that matched or mismatched events described by the sentence. Contrary to previous findings (Crain, S., Ni, W., & Conway, L. (1994). Learning, parsing and modularity. In C. Clifton, L. Frazier, & K. Rayner (Eds.), Perspectives on sentence processing. Hillsdale, NJ: Lawrence Erlbaum; Philip, W., & Lynch, E. (1999). Felicity, relevance, and acquisition of the grammar of every and only. In S. C. Howell, S. A. Fish, & T. Keith-Lucas (Eds.), Proceedings of the 24th annual Boston University conference on language development. Somerville, MA: Cascadilla Press) we found that young children predominantly made errors by failing to process contrast information rather than errors in which they failed to use syntactic information to restrict the scope of the particle. Experiment 3 replicated these findings with pre-schoolers.     

Perception of motion affects language processing

Recently developed accounts of language comprehension propose that sentences are understood by constructing a perceptual simulation of the events being described. These simulations involve the re-activation of patterns of brain activation that were formed during the comprehender's interaction with the world. In two experiments we explored the specificity of the processing mechanisms required to construct simulations during language comprehension. Participants listened to (and made judgments on) sentences that described motion in a particular direction (e.g. "The car approached you"). They simultaneously viewed dynamic black-and-white stimuli that produced the perception of movement in the same direction as the action specified in the sentence (i.e. towards you) or in the opposite direction as the action specified in the sentence (i.e. away from you). Responses were faster to sentences presented concurrently with a visual stimulus depicting motion in the opposite direction as the action described in the sentence. This suggests that the processing mechanisms recruited to construct simulations during language comprehension are also used during visual perception, and that these mechanisms can be quite specific.     

Word Detection in Sung and Spoken Sentences in Children With Typical Language Development or With Specific Language Impairment

Background: Previous studies have reported that children score better in language tasks using sung rather than spoken stimuli. We examined word detection ease in sung and spoken sentences that were equated for phoneme duration and pitch variations in children aged 7 to 12 years with typical language development (TLD) as well as in children with specific language impairment (SLI ), and hypothesized that the facilitation effect would vary with language abilities. Method: In Experiment 1, 69 children with TLD (7–10 years old) detected words in sentences that were spoken, sung on pitches extracted from speech, and sung on original scores. In Experiment 2, we added a natural speech rate condition and tested 68 children with TLD (7–12 years old). In Experiment 3, 16 children with SLI and 16 age-matched children with TLD were tested in all four conditions. Results: In both TLD groups, older children scored better than the younger ones. The matched TLD group scored higher than the SLI group who scored at the level of the younger children with TLD . None of the experiments showed a facilitation effect of sung over spoken stimuli. Conclusions: Word detection abilities improved with age in both TLD and SLI groups. Our findings are compatible with the hypothesis of delayed language abilities in children with SLI , and are discussed in light of the role of durational prosodic cues in words detection.     

How reading acquisition changes children’s spoken language network

To examine the influence of age and reading proficiency on the development of the spoken language network, we tested 6- and 9-years-old children listening to native and foreign sentences in a slow event-related fMRI paradigm. We observed a stable organization of the peri-sylvian areas during this time period with a left dominance in the superior temporal sulcus and inferior frontal region. A year of reading instruction was nevertheless sufficient to increase activation in regions involved in phonological representations (posterior superior temporal region) and sentence integration (temporal pole and pars orbitalis). A top-down activation of the left inferior temporal cortex surrounding the visual word form area, was also observed but only in 9 year-olds (3 years of reading practice) listening to their native language. These results emphasize how a successful cultural practice, reading, slots in the biological constraints of the innate spoken language network.     

Spatial language influences memory for spatial scenes

Does language influence recognition for spatial scenes? In Experiments 1 and 2, participants viewed ambiguous pictures, with or without spatial sentences. In a yes—no recognition task, only the spatial sentences group made more false alarms toward the center of the spatial category than in the other direction; three other comparison groups showed no such tendency. This shift toward the core of the semantic category suggests that spatial language interacted with perceptual information during encoding. In Experiment 3, we varied the materials to test the interactive encoding account against a separate encoding account in which separately stored sentences are accessed during picture recognition. The results support the interactive encoding account in which spatial language influences the encoding and memory of spatial relations.     

汉语确定性无界否定句模拟加工的时间进程

以往关于否定句理解的心理模拟过程是一步完成的还是两步完成的, 结果并不一致, 这与各个研究使用的实验材料类型有关。本研究以汉语确定性无界否定句为研究对象, 采用句-图匹配范式和2(句子类型：汉语确定性无界肯定句和汉语确定性无界否定句) × 2 (图片描绘事物状态与句子描述的事物状态的匹配类型：匹配与不匹配)被试内实验设计, 以对图片判断的反应时和正确率为指标, 探讨其理解的早期(250 ms)、中期(750 ms)和晚期(1500 ms)的心理模拟过程。研究结果表明：确定性无界否定句理解的心理模拟是分两步进行的, 首先模拟的是否定句的被否定状态, 随着加工时间的推进, 完成了对确定性无界否定句的第二步模拟即对实际状态的模拟。但是, 比其他类型否定句的模拟过程所用的时间要短, 即在阅读理解的中期就完成了心理模拟。     

Reading span task performance,linguistic experience,and the processing of unexpected syntactic events

Accounts of individual differences in online language processing ability often focus on the explanatory utility of verbal working memory, as measured by reading span tasks. Although variability in reading span task performance likely reflects individual differences in multiple underlying traits, skills, and processes, accumulating evidence suggests that reading span scores also reflect variability in the linguistic experiences of an individual. Here, through an individual differences approach, we first demonstrate that reading span scores correlate significantly with measures of the amount of experience an individual has had with written language (gauged by measures that provide “proxy estimates” of print exposure). We then explore the relationship between reading span scores and online language processing ability. Individuals with higher reading spans demonstrated greater sensitivity to violations of statistical regularities found in natural language—as evinced by higher reading times (RTs) on the disambiguating region of garden-path sentences—relative to their lower span counterparts. This result held after statistically controlling for individual differences in a non-linguistic operation span task. Taken together, these results suggest that accounts of individual differences in sentence processing can benefit from a stronger focus on experiential factors, especially when considered in relation to variability in perceptual and learning abilities that influence the amount of benefit gleaned from such experience.     

The role of perceptual similarity of the task environments in children’s perseverative responding

Perseverative responding is often seen in children’s performance in a variety of contexts. One such context is symbolic comprehension in which 2- and 2½-year-olds demonstrate difficulty in appreciating the association of symbols (pictures and scale models) and their referents and show a high proportion of perseverative responding. Representational-level explanations of perseveration were explored in the current studies via examination of the impact of perceptual similarity of visual environments across trials. Across two experiments, children saw either a picture (Experiment 1) or a scale model (Experiment 2) of a hiding location of a room and were then encouraged to recover the toy from an identical room. Manipulating the perceptual similarity of the environments across successive trials affected performance and perseverative responding. These results highlight the critical role played by perceptual information not only in symbolic tasks but also in many other tasks and have important implications for theories of perseveration.     

Automatic perceptual simulation of first language meanings during second language sentence processing in bilinguals

Research supports the claim that, when understanding language, people perform mental simulation using those parts of the brain which support sensation, action, and emotion. A major criticism of the findings quoted as evidence for embodied simulation, however, is that they could be a result of conscious image generation strategies. Here we exploit the well-known fact that bilinguals routinely and automatically activate both their languages during comprehension to test whether this automatic process is, in turn, modulated by embodied simulatory processes. Dutch participants heard English sentences containing interlingual homophones and implying specific distance relations, and had to subsequently respond to pictures of objects matching or mismatching this implied distance. Participants were significantly slower to reject critical items when their perceptual features matched said distance relationship. These results suggest that bilinguals not only activate task-irrelevant meanings of interlingual homophones, but also automatically simulate these meanings in a detailed perceptual fashion. Our study supports the claim that embodied simulation is not due to participants' conscious strategies, but is an automatic component of meaning construction.     

语言理解中情绪的动态模拟

采用混合实验设计, 句子-图片匹配范式和动态过程再现范式, 以包含静态情绪和动态变化情绪信息的句子为实验材料, 以被试对图片的反应时间为因变量指标, 通过不同时间间隔的设置以及不同的实验程序, 探讨了语言理解中静态情绪和动态情绪的心理模拟及其时间进程。结果表明：(1)被试对情绪的心理模拟与句中的语言描述一致, 情绪语境能够激活被试的经验模拟。(2)被试在句子理解加工的初期就能迅速、即时地模拟静态情绪信息, 且这种模拟具有一定的稳定性和持久性。(3)被试可以对动态变化的情绪进行动态的心理模拟, 不过这种模拟所需的时间要长, 是在句子理解加工的后期完成的。被试对不同情绪变化方向的句子模拟受到原情绪状态的影响, 当阅读由消极情绪向积极情绪变化的句子时, 由于被试固着于原来的消极情绪而发生了模拟积极情绪的困难。