Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search

When we see combinations of text and graphics, such as photographs and their captions in printed media, how do we compare the information in the two components? Two experiments used a sentence-picture verification task in which statements about photographs of natural scenes were read in order to make a true/false decision about the validity of the sentence, and in which eye movements were recorded. In Experiment 1 the sentence and the picture were presented concurrently, and objects and words could be inspected in any order. In Experiment 2 the two components were presented one after the other, either picture first or sentence first. Fixation durations on pictures were characteristically longer than those on sentences in both experiments, and fixations on sentences varied according to whether they were being encoded as abstract propositions or as coreferents of objects depicted in a previously inspected picture. The decision time data present a difficulty for existing models of sentence verification tasks, with an inconsistent pattern of differences between true and false trials.     

句法结构和动词重复对汉语句子口语产生中句法启动效应的影响

研究中采用启动范式和图片描述任务, 利用句法选择比率和句子产生潜伏期的两个指标, 考察了启动句句法结构、动词是否相同和延时对汉语口语句子产生中句法启动效应的影响。结果发现在选择比率上, 句法结构产生的启动效应随延时变化保持稳定, 由启动句和目标句动词相同导致的启动效应的提高量(词汇增强效应)快速衰退, 首次为句子产生过程的句法选择阶段和计划阶段提供了证据。在句子产生潜伏期上, 仅发现启动句和目标句句法结构相同时缩短了句子产生潜伏期, 这可能是由于启动句句法结构激活水平的提高, 使得说话者在目标句产生的句法计划和选择阶段都更快引起的。选择阶段的加工影响句法选择比率, 而选择阶段和计划阶段共同影响了句子产生的潜伏期, 实验结果支持了两阶段竞争理论的观点。     

新颖词汇学习中大脑激活模式的fMRI研究

该研究利用功能磁共振脑成像(fMRI)技术,比较在新颖语言学习和母语学习过程中大脑皮层参与的异同。实验设置德语学习和汉语学习两个条件,采用跨通道联结学习范式,要求被试在听到听觉词汇后,按键判断屏幕上所呈现的图片是否与所听词汇匹配。行为数据显示,被试在六段实验后基本掌握了德语语音和语义之间的联结关系;脑成像数据显示,两种学习任务都激活了语言加工的主要脑区,而德语词汇学习过程相比汉语参照条件对应的颞叶激活强度与范围有较大的差异,并且比汉语条件表现出更显著的前额叶区域激活。     

Clauses and the semantic representation of words

When a sentence with more than one clause is processed, words of the first clause become less available for recall or recognition once the clause boundary has been passed. One common interpretation of this observation is that the representation of a given word shifts from a predominantly surface form (e.g., phonological or lexical) to a semantic form, after the clause boundary. Two experiments that test that interpretation are reported. In the first experiment, two-clause spoken sentences were followed by spoken word or picture probes. Pictures were intended to provide a fast semantic match to probed words, but a slow surface match. Although response times exhibited a robust main effect of the clause position of the probe, no interaction with probe type was found. Similar results were obtained in a second experiment that compared pictures with written word probes. The faster response to picture than to word probes in the second experiment indicates that subjects did not covertly name the picture, but made a semantic match (as intended). These observations suggest the following reinterpretation of earlier sentence memory experiments: Words in the most recent clause of a sentence are more available than words in an earlier clause because their semantic representations are more active, not because their surface representations are more active.     

The activation of word class information during speech production

In 5 picture-word interference experiments the activation of word class information was investigated. The first experiment, in which subjects used bare nouns to describe the pictures, failed to reveal any interference effect of noun distractor words as opposed to closed-class distractor words. In the next 4 experiments the pictures were named by using a definite determiner and the noun completing a sentence fragment. The data demonstrate that noun distractors interfere more strongly with picture naming than do non-noun distractors. This held for both visual and auditory presentation of the distractor words. The interference effect showed up in a time window where semantic interference can usually be observed, supporting the assumption that at an early stage of lexical access semantic and syntactic activation processes overlap.     

Generation of a pictorial code

In Experiments I and II direct measures were obtained of the time required to convert a picture or a sentence to a report of visualization or to a simple drawing, latencies were faster for pictures than for sentences and were affected by the surface form of the sentence. In Experiment Ill, Ss matched pictures or sentences against a test picture under conditions of simultaneous or successive presentation. “Yes” and “No” response times were affected by sentence form under both conditions, although the difference between sentence-picture and picture-picture comparisons was virtually eliminated under the successive condition. In Experiment IV, Ss held a sentence or picture in memory over a 1,500-msec interval at the end of which a test picture or an instruction to draw the design was presented. Construction latencies were shorter in response to pictures than sentences, and there were effects of sentence form on verification time. Some implications of these results are discussed.     

Perceptual simulation in developing language comprehension

We tested an embodied account of language proposing that comprehenders create perceptual simulations of the events they hear and read about. In Experiment 1, children (ages 7–13 years) performed a picture verification task. Each picture was preceded by a prerecorded spoken sentence describing an entity whose shape or orientation matched or mismatched the depicted object. Responses were faster for matching pictures, suggesting that participants had formed perceptual-like situation models of the sentences. The advantage for matching pictures did not increase with age. Experiment 2 extended these findings to the domain of written language. Participants (ages 7–10 years) of high and low word reading ability verified pictures after reading sentences aloud. The results suggest that even when reading is effortful, children construct a perceptual simulation of the described events. We propose that perceptual simulation plays a more central role in developing language comprehension than was previously thought.     

Centering on figurative features during the comprehension of sentences describing scenes

Summary Subjects were asked to read sentences which described scenes containing objects. In the scene described in each sentence, a specific part of a particular object was necessarily implied as having an important role. The object was named but none of its parts were. The assumption was that the subjects, while processing the sentence and immediately afterward, would cognitively center on the important part, as a function of the context created by the sentence. Immediately after reading the sentence, the subjects were probed with a picture of either the important part, or of an unimportant part of the object. Judgments of the compatibility of this picture probe with the sentence were faster for pictures of important parts than for pictures of other parts. This was taken as supporting the hypothesis of cognitive centration. In a second experiment, in which words were used as probes instead of pictures, a purely verbal process to account for the results was ruled out. In a third experiment, subjects were given instructions to intentionally form visual images of the scenes described by the sentences. In this case, overall response times to the picture probes were shorter than in the absence of such instructions, but this decrease was greater for pictures of unimportant parts. This finding was interpreted as showing that imagery instructions increase the rate of activation of features to varying degrees as a function of the previous level of activation.     

Reading and listening comprehension in Broca's aphasia: Lexical versus syntactical errors

Ten agrammatic Broca's aphasics were presented with a series of four picture plates together with a spoken or written sentence stimulus. All sentence stimuli were of the structure, the + N + is/are + V + ing + the + N. The four pictures on each stimulus plate represented (a) the correct response, (b) a reversal of the stimulus sentence subject and object, (c) a change in the number of the subject of the stimulus sentence, and (d) a change in one of the major lexical items of the stimulus sentence. Subjects selected the correct picture most often. When they erred, they usually selected a subject-object reversal. Number errors were less frequent, and the patients seldom selected a change in major lexical item. This pattern occurred with both written and spoken sentences. These results were interpreted as reflecting the dependence of agrammatic Broca's aphasics on the semantic interpretation of the lexicon for decoding sentences in the face of deficits in syntactical-grammatical interpretation, irrespective of comprehension modality.     

汉语口语句子产生中句法启动效应的眼动研究

研究利用眼动技术考察了汉语口语句子产生中的句法启动效应及其眼动模式。实验采用双宾语句(主语+谓语+间接宾语+直接宾语)、介宾语句(主语+谓语+直接宾语+间接宾语)和简单句三种启动句类型,听觉呈现启动句后呈现图片要求被试用句子进行描述,目标句类型为双宾语句或介宾语句。结果发现,启动句类型影响了目标句的句法选择及对图片中不同客体的注视时间比率及注视顺序。无论产生的目标句是否与启动句句法结构一致,双宾语句启动下对间接宾语的客体注视时间比率更高,且更多地优先注视间接宾语,介宾语句启动下对直接宾语的客体注视时间比率更高且更多地优先注视直接宾语,而在简单句启动条件下,眼动注视模式则会受到所产生目标句句法结构的影响。行为结果和眼动指标都支持了两阶段理论中结构选择和内容计划两阶段的观点。     

Pictorial priming and cued recall in Alzheimer's and Huntington's disease

A picture fragment test was used to compare the priming and cued recall performances of patients with Huntington's disease (HD), patients with dementia of the Alzheimer type (DAT), and neurologically intact normal control (NC) subjects. On the pictorial priming test, subjects were asked to say "the first thing you think of" when shown incompletely drawn pictures, half of which were previously exposed to the subject during an unrelated naming task. Normal controls and HD patients, but not DAT patients, demonstrated similar increases in their ability to identify fragmented versions of previously seen pictures relative to novel pictures. These results are consistent with the previously observed pattern of preserved and impaired verbal priming ability in HD and DAT patients, respectively. The NC subjects, as expected, also demonstrated better performance on the cued recall than on the priming version of the picture fragment test, whereas the HD patients evidenced the opposite relationship on these two tasks and DAT patients were found to be equally impaired on both tests. This finding provides further support for the notion that HD patients' memory impairment is characterized primarily by an inability to initiate systematic retrieval strategies.     

Partial picture effects on children's memory for sentences containing implicit information

Two experiments were conducted examining the effects of partial picture adjuncts on young children's coding of information that was implied in sentences. In the two most critical conditions of these studies, subjects were presented sentences specifying a subject, an action, and a direct object with the instrument used to carry out the action not specified in the sentence (e.g., The workman dug a hole in the ground). Implicit-sentence-only subjects received only the sentences, whereas the implicit sentence + partial picture subjects also viewed a partial picture depicting the action in the sentence minus the implied instrument. The main hypothesis was that subsequent recall of the sentences given the implied instrument as a cue would be facilitated by the partial pictures provided at study, since they would lead the children to infer the instrument. That occurred with 6- to 7-year-old children, but not with preschool children. Consistent with the conclusion that the partial pictures prompted 6- to 7-year-olds to infer the instruments, implicit sentence + partial picture subjects recalled as much as subjects in two other conditions, one in which subjects were explicitly told the instruments at study and one in which subjects saw the instruments depicted in pictures at study. In contrast, preschool subjects who heard explicit sentences containing the instruments outperformed subjects who heard implicit sentences even when the implicit sentences were accompanied by pictures depicting the instruments. This failure of complete pictures to facilitate preschoolers' recall of information implied in sentences contrasts with the many demonstrations of prose learning facilitation when picture and sentence contents explicitly and completely overlap. In summary, there were developmental differences in whether (a) partial pictures significantly facilitated inferencing (and subsequent cued recall) and (b) complete pictures containing information not explicitly stated in sentences promoted cued recall of the sentences.     

Spoken syntax in children with acquired unilateral hemisphere lesions

The spoken syntax of eight left hemisphere lesioned and eight right hemisphere lesioned children were compared to matched controls. The children's lesions were acquired between 0.08 and 6.17 years of age (mean = 1.33 years), and at the time of testing they were between 1.67 and 8.15 years of age (mean = 4.19). Based on analyses of spontaneous language samples, left hemisphere lesioned subjects performed more poorly than did their controls on most measures of simple and complex sentence structure. In contrast right lesioned subjects performed similarly to their controls on these measures, except for a tendency to make more errors in simple sentence structures. These findings provide further evidence that the left and right hemispheres are not comparable in supporting syntactic abilities.     

Speech disfluencies in nonstutterers Syntactic complexity and production task effects

The differences in the number of disfluencies of 3-, 5-, and 7-year-old nonstuttering children were investigated as syntactic complexity was varied in three different syntactic constructions: 1) simple affirmative declarative with copula + ing (SAAD), 2) future (FUT), and 3) passive (PAS) within a sentence imitation and a sentence-modeling task. The subjects repeated 30 sentences from the imitation task stimuli and produced response constructions from 30 pictures from the modeling task stimuli. The 3-year-olds exhibited significantly more disfluencies than the 5-year-olds, and the 5-year-olds produced significantly more disfluencies than the 7-year-olds. All three age groups produced significantly more disfluencies on the modeling task than on the imitation task. A significant complexity effect was demonstrated for the PAS construction. Results are compared to previous research findings and discussed with reference to hypothesized linguistic complexity and task difficulty.     

Working Memory and On-Line Sentence Comprehension in Patients with Alzheimer's Disease

We examined the ability of patients with dementia of the Alzheimer's type (DAT) and normal controls to perform a sentence acceptability judgment task that required determining the referent for a reflexive pronoun. Performance on three different sentence types that differed in terms of syntactic complexity was assessed. Subjects performed the task alone and under two different dual-task conditions which required continuous, externally paced responses. DAT patients were more affected than controls by the dual-task conditions, but were not disproportionately impaired on the more complex sentence types. The failure of DAT patients to he disproportionately affected on the most complex sentence types in the dual-task conditions provides evidence for the separation of the processing resources that are used in sentence comprehension from those involved in other tasks.