Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search

When we see combinations of text and graphics, such as photographs and their captions in printed media, how do we compare the information in the two components? Two experiments used a sentence-picture verification task in which statements about photographs of natural scenes were read in order to make a true/false decision about the validity of the sentence, and in which eye movements were recorded. In Experiment 1 the sentence and the picture were presented concurrently, and objects and words could be inspected in any order. In Experiment 2 the two components were presented one after the other, either picture first or sentence first. Fixation durations on pictures were characteristically longer than those on sentences in both experiments, and fixations on sentences varied according to whether they were being encoded as abstract propositions or as coreferents of objects depicted in a previously inspected picture. The decision time data present a difficulty for existing models of sentence verification tasks, with an inconsistent pattern of differences between true and false trials.     

Understanding metaphoric sentences in the two cerebral hemispheres

Previous studies indicate that the right hemisphere (RH) has a unique role in maintaining activation of metaphoric single word meanings. The present study investigated hemispheric asymmetries in comprehending metaphoric word meanings within a sentence context. Participants were presented with incomplete priming sentences followed by (literally) true, false, or metaphoric lateralized target words and were asked to decide whether each sentence is literally true or false. Results showed that responses to metaphoric sentences were slower and less accurate than to false sentences when target words were presented to the right visual field (RVF)-LH as well as to the left visual field (LVF)-RH. This suggests that the understanding of lexical metaphors within a sentence context involves LH as well as RH processing mechanisms and that the role of each hemisphere in processing nonliteral language is flexible and may depend on the linguistic task at hand.     

Brain activity varies with modulation of dynamic pitch variance in sentence melody

Fourteen native speakers of German heard normal sentences, sentences which were either lacking dynamic pitch variation (flattened speech), or comprised of intonation contour exclusively (degraded speech). Participants were to listen carefully to the sentences and to perform a rehearsal task. Passive listening to flattened speech compared to normal speech produced strong brain responses in right cortical areas, particularly in the posterior superior temporal gyrus (pSTG). Passive listening to degraded speech compared to either normal or flattened speech particularly involved fronto-opercular and subcortical (Putamen, Caudate Nucleus) regions bilaterally. Additionally the Rolandic operculum (premotor cortex) in the right hemisphere subserved processing of neat sentence intonation. As a function of explicit rehearsing sentence intonation we found several activation foci in the left inferior frontal gyrus (Broca's area), the left inferior precentral sulcus, and the left Rolandic fissure. The data allow several suggestions: First, both flattened and degraded speech evoked differential brain responses in the pSTG, particularly in the planum temporale (PT) bilaterally indicating that this region mediates integration of slowly and rapidly changing acoustic cues during comprehension of spoken language. Second, the bilateral circuit active whilst participants receive degraded speech reflects general effort allocation. Third, the differential finding for passive perception and explicit rehearsal of intonation contour suggests a right fronto-lateral network for processing and a left fronto-lateral network for producing prosodic information. Finally, it appears that brain areas which subserve speech (frontal operculum) and premotor functions (Rolandic operculum) coincidently support the processing of intonation contour in spoken sentence comprehension.     

Effects of paraphrase and schema on intrusions,normalizations, and recall of thematic prose

The present experiment investigated the effects of task processing, historical knowledge of the subject matter, and retention interval on normalization rates, intrusions, and recall of thematic prose. Learners were given 16 sentences about the life and times of either Adolf Hitler or a fictitious character. Half the sentences were historically incorrect for Hitler but were not verifiable in terms of the fictitious character. Learners either paraphrased or copied the sentences verbatim and attempted to recall the sentences at either a zero retention interval or after 48 hours. Normalization and intrusion rates were strongly related to story type and retention interval, while overall sentence recall was influenced by processing task and retention interval. The results are discussed in terms of both the reconstructive schema model and the depth of processing hypothesis.     

The relation between comprehending and remembering some complex sentences

This research concerns the distinction between processes in sentence comprehension and those in sentence memory. Comprehension was monitored by timing subjects while they decided whether a sentence was true or false of their knowledge of the world. The memory process was tapped by examining subjects' incidental memory for the sentences they had previously verified. The verification latencies indicated that at the time of comprehension, sentences likeIt is true that a fire isn't cold were often recoded into an equivalent affirmative form, namely,It is true that a fire is hot. However, negative sentences likeIt isn't true that a fire is cold were not recoded during comprehension. Recoding in recall occurred only for those types of sentences that were recoded at the time of comprehension.     

Dissociable patterns of brain activity during comprehension of rapid and syntactically complex speech: evidence from fMRI

Sentence comprehension is a complex task that involves both language-specific processing components and general cognitive resources. Comprehension can be made more difficult by increasing the syntactic complexity or the presentation rate of a sentence, but it is unclear whether the same neural mechanism underlies both of these effects. In the current study, we used event-related functional magnetic resonance imaging (fMRI) to monitor neural activity while participants heard sentences containing a subject-relative or object-relative center-embedded clause presented at three different speech rates. Syntactically complex object-relative sentences activated left inferior frontal cortex across presentation rates, whereas sentences presented at a rapid rate recruited frontal brain regions such as anterior cingulate and premotor cortex, regardless of syntactic complexity. These results suggest that dissociable components of a large-scale neural network support the processing of syntactic complexity and speech presented at a rapid rate during auditory sentence processing.     

When people are more logical under cognitive load: dual task impact on scalar implicature

The present study introduces dual task methodology to test opposing psychological processing predictions concerning the nature of implicatures in pragmatic theories. Implicatures routinely arise in human communication when hearers interpret utterances pragmatically and go beyond the logical meaning of the terms. The neo-Gricean view (e.g., Levinson, 2000) assumes that implicatures are generated automatically whereas relevance theory (Sperber & Wilson, 1986/1995) assumes that implicatures are effortful and not automatic. Participants were presented a sentence verification task with underinformative sentences that have the potential to produce scalar implicatures like Some oaks are trees. Depending on the nature of the interpretation of Some (logical or pragmatic) the sentence is judged true or false. Executive cognitive resources were experimentally burdened by the concurrent memorization of complex dot patterns during the interpretation process. Results showed that participants made more logical and fewer pragmatic interpretations under load. Findings provide direct support for the relevance theory view.     

无关言语对阅读的干扰基于内容还是过程？——来自眼动研究的证据

采用眼动追踪技术,考察无关言语对句子材料相同的自然阅读和校对阅读的影响,探讨无关言语对阅读的干扰机制是基于内容还是过程。结果发现,有意义言语显著干扰了正常的阅读行为,而无意义言语与无背景音条件不存在显著差异。并且背景言语类型与任务类型的交互作用显著,表现为言语的语义成分只干扰了自然阅读,而对校对阅读没有影响。实验结果支持过程干扰假说。     

SEMANTIC AND PHONOLOGICAL CODING PROCESSES IN SKILLED AND LESS SKILLED READERS' READING OF TEXT

Abstract

Two experiments examine the memory coding processes of skilled and less skilled readers during the reading of connected text. In experiment 1, students read several paragraphs which required a lexical decision about an underlined letter string within a sentence. Underlined letter strings were either synonyms, repeated words, or control words in reference to items in the sentence. Students were later asked to recall words related to their lexical decision, as well as verify the occurrence of sentences from the text. Skilled readers recalled more synonyms than poor readers, whereas no differences emerged between groups in their recall of other types of words related to the lexical task or for the verification of sentences. Experiment 2 procedures were similar to Experiment 1, except that synonyms were replaced with homophones and the sentence verification task included phrases related to the homophones. When compared to less skilled readers, skilled readers recalled more homophones and repeated words, but were more likely to be disrupted in correct verification of sentences with homophones. Taken together, the experiments suggest that along with phonological coding, semantic processing contributes an important amount of variance to deficiencies in the reading of connected text.     

Semantic involvement in the lexical and sentence processing of Japanese kanji

This study examined how skilled Japanese readers activate semantic information when reading kanji compound words at both the lexical and sentence levels. Experiment 1 used a lexical decision task for two-kanji compound words and nonwords. When nonwords were composed of kanji that were semantically similar to the kanji of real words, reaction times were longer and error rates were higher than when nonwords had kanji that were not semantically similar. Experiment 2 used a proofreading task (detection of kanji miscombinations) for the same two-kanji compound words and nonwords at the sentence level. In this task, semantically similar nonwords were detected faster than dissimilar nonwords, but error rates were much higher for the semantically similar nonwords. Experiment 3 used a semantic decision task for sentences with the same two-kanji compound words and nonwords. It took longer to detect semantically similar nonwords than dissimilar nonwords. This indicates that semantic involvement in the processing of Japanese kanji produces different effects, depending on whether this processing is done at the lexical or sentence level, which in turn is related to where the reader's attention lies.     

Interference and facilitation effects of primes upon verification processes

Priming effects on sentence verification were investigated. The semantic relation of the prime and the probe, and the interval between prime and probe presentation (SOA), were varied for both ambiguous and unambiguous sentences. Reaction time to decide that a sentence was true or false was longer if the preceding prime was a word that was unrelated to the probe than if the prime was the word “blank.” In contradiction of Posner and Snyder’s (1975) claim that conscious processes develop slowly, this result was found at SOAs as short as 250 msec. Verification performance was facilitated for both sentence types when the prime word was the first of the to-be-presented probe sentence, but the magnitude of the facilitation effects depended upon the truth value of the probe, the associative strength of the subject and predicate of the probe sentence, and upon SOA. These findings indicate that priming affects the processing of relations among concepts in semantic memory, as well as the encoding of the probe.     

Sentence processing: Evidence against the serial,independent stage assumption

The assumption that the processing stages involved in sentence comprehension are serially and independently executed was tested in two separate experiments using two new sentence types. One experiment was a sentence-picture verification task, the other was not. In Experiment I, subjects viewed sentences such as It's false that the dots are red and indicated, by pressing the appropriate response key, which of two colors would make the sentence true. In Experiment II, subjects verified sentences such as It isn't true that the dots aren't red against pictures of colored dots. On the strength of Sternberg's additive factor method, present findings challenge the validity of the serial, independent stage assumption, and results are alternatively discussed in terms of a capacity sharing model.This research was supported by PHS research grant MH 23401 to Sam Glucksberg, principal investigator.     

The roles of sequencing and verbal working memory in sentence comprehension deficits in Parkinson's disease

Studies of sentence comprehension deficits in Parkinson's disease (PD) patients suggest that language processing involves circuits connecting subcortical and cortical regions. Anatomically segregated neural circuits appear to support different cognitive and motor functions. To investigate which functions are implicated in PD comprehension deficits, we tested comprehension, verbal working memory span, and cognitive set-switching in a non-linguistic task in 41 PD patients; we also obtained speech measurements reflecting motor sequencing processes that may be involved in articulatory rehearsal within working memory. Comprehension of sentences with center-embedded or final relative clauses was impaired when they could not be understood from lexical semantic content alone. Overall comprehension error rates correlated strongly with impaired set-switching and significantly with reduced working memory span and speech motor sequencing deficits. Correlations with comprehension of different sentence structures indicate that these impairments do not represent a single deficit; rather, PD comprehension deficits appear to arise from several independent mechanisms. Deficits in cognitive set-switching or underlying inhibitory processes may compromise the ability to process relative clauses. Deficits in verbal working memory appear to impair comprehension of long-distance dependencies. Speech sequencing correlated with neither set-switching nor verbal working memory span, consistent with their being supported by independent, segregated cortico-subcortical circuits.     

Insight into a logical relation

Two experiments are reported which aimed to investigate factors affecting the gain of insight into the logical relation of implication. In the first experiment, subjects had to make a series of inferences about either a conditional sentence or a quantified sentence, both of which had the same underlying logical form. Under one condition the sentences had to be proved true, and under another condition, false. Proving a sentence false facilitated gain of insight, but the linguistic form of the sentence exerted no significant effect on the main dependent variable. In the second experiment, implication was not expressed as a sentence but was inherent in the structure of the task. The experimental material differed in complexity and allowed the cognitive load imposed on the subject to be varied. Results suggested that insight was not all-or-none. It was spontaneously gained when the material was simple, but temporarily lost when it was complex.     

Context and Complexity in Incremental Sentence Interpretation: An ERP Study on Temporal Quantification

The present event-related potential (ERP) study used picture–sentence verification to investigate the neurolinguistic correlates of the online processing of compositional-semantic information. To this end, we examined context effects on sentences involving temporal adverbial quantification likeJana war jeden Morgen schwimmen an den Arbeitstagen (“Jana went for a swim every morning during the working week”). We tested whether the conceptual complexity associated with quantifying over time intervals leads to delayed predictions regarding the upcoming words in a sentence. The present study replicated previous results relating to quantification over individuals, which are conceptually less complex than time intervals. Analogous to previous studies, false vs. true sentences elicited an N400 whenever contextual cues did not permit a potential revision of a locally assigned truth value. The present results are compatible with an approach under which contextual cues are immediately considered for predicting how a sentence continues. The fact that the contextual complexity did not lead to processing delays indicates that the processing system quickly abstracts away from the conceptual complexity associated with the linguistic input if such an abstraction is possible.     

The role of imagery in the evaluation of sentences

Reaction time was the dependent variable in this sentence verification experiment. Simple sentences, which were either true or false, were constructed such that in some cases it was easy to form an image of the sentence, and in other cases forming an image was difficult. Ss' ratings were employed to select low- and high-imagery sentences. It was found that high-imagery sentences could be verified more rapidly than low-imagery sentences. Instructions which did not mention imagery were given to some Ss while others were specifically asked to employ imagery, but the reaction times of these two groups of Ss did not differ. It was concluded that imagery facilitates the process of sentence verification and that models of this process which restrict themselves to purely semantic operations are incomplete.     

Syntactic presupposition in sentence comprehension

Two experiments investigated the role of syntactic presupposition in sentence comprehension. In Experiment I subjects verified cleft, pseudocleft and factive complement sentences with respect to preceding context paragraphs, which contradicted either the assertion or the presupposition of the target sentence. Subjects took significantly longer to verify sentences with false presuppositions than sentences with false assertions. In Experiment II subjects verified cleft and pseudocleft sentences with respect to subsequently presented pictures. Once again, verification times for sentences with false presuppositions were significantly longer than verification times for sentences with false assertions. It was argued that these findings are more adequately explained by a “structural” hypothesis, than in terms of strategies designed to locate given and new information.     

The effects of processing requirements on neurophysiological responses to spoken sentences

Ten English speaking subjects listened to sentences that varied in sentential constraint (i.e., the degree to which the context of a sentence predicts the final word of that sentence) and event-related potentials (ERPs) were recorded during the presentation of the final word of each sentence. In the Control condition subjects merely listened to the sentences. In the Orthographic processing condition subjects merely listened to the sentences. In the Orthographic processing condition subjects decided, following each sentence, whether a given letter had been present in the final word of the preceding sentence. In the Phonological processing condition the subjects judged whether a given speech sound was contained in the terminal word. In the Semantic processing condition subjects determined whether the final word was a member of a given semantic category. A previous finding in the visual modality that the N400 component was larger in amplitude for low constraint sentence terminations than for high was extended to the auditory modality. It was also found that the amplitude of a N200-like response was similarly responsive to contextual constraint. The hypothesis that N400 amplitude would vary significantly with the depth of processing of the terminal word was not supported by the data. The "N200" recorded in this language processing context showed the classic frontocentral distribution of the N200. The N400 to spoken sentences had a central/centroparietal distribution similar to the N400 in visual modality experiments. It is suggested that the N400 obtained in these sentence contexts reflects an automatic semantic processing of words that occurs even when semantic analysis is not required to complete a given task. The cooccurrence and topographical dissimilarity of the "N200" and N400 suggest that the N400 may not be a delayed or a generic N200.     

Building syntactic structures in speaking: a bilingual exploration

In a series of three experiments we investigated syntactic priming using a sentence recall task. Participants read and memorized a target sentence for later recall. After reading a prime sentence and engaging in a distraction task, they were asked to produce the target sentence aloud. Earlier investigations have shown that this task is sensitive to a syntactic priming effect. That is, the syntactic form of the prime sentence sometimes influences the syntactic form of the recalled target. In this paper we report on a variation on this task, using Spanish-English bilingual participants. In the first two experiments we replicated the prepositional phrase priming effect using English target sentences and Spanish prime sentences. In the final experiment we investigated two additional syntactic forms, using Spanish target sentences and English prime sentences. Implications for models of syntax generation and bilingual speech production are discussed.     

Linguistic structures and mental operations

The internal representations and mental operations used in verifying a sentence against a pictorial referent are examined in an experiment where the location of the false constituent (subject, verb, or object) varied. The latencies for false sentences were ordered: verb, grammatical subject, grammatical object, with verb mismatches being detected the fastest. An information-processing analysis indicated that both comparison and search operations are employed, and that sentences are represented, in general, as a list of propositions with case-relational information. Context and task demands impose order on the list and lead to different operations on these structures. The depth of processing sentences varies from simple lexical comparisons up to full encoding and usage of syntactic and semantic features and case relations.