Disfluencies and human language comprehension

Spoken language contains disfluencies, which include editing terms such as uh and um as well as repeats and corrections. In less than ten years the question of how disfluencies are handled by the human sentence comprehension system has gone from virtually ignored to a topic of major interest in computational linguistics and psycholinguistics. We discuss relevant empirical findings and describe a computational model that captures how disfluencies influence parsing and comprehension. The research reviewed shows that the parser, which presumably evolved to handle conversations, deals with disfluencies in a way that is efficient and linguistically principled. The success of this research program reinforces the current trend in cognitive science to view cognitive mechanisms as adaptations to real-world constraints and challenges.     

Social status and language comprehension

Abstract.— Previous research on differences in language use in relation to social status have dealt almost exclusively with language production. The present study investigated the comprehension of connected discourse by 116 adult Danes from two different socio-economic status groups, high status (HSES) and low status (LSES). On a task at the lexical level of comprehension (choosing between alternatives to two words in each of four texts) there appeared no differences between HSES and LSES subjects. At the structural level a difference ( p <0.05) was found in drawing conclusions from the texts, but the groups nevertheless agreed completely ( p <0.05) in underlining important segments of the texts. Thus the symmetry between production and comprehension hypothesized by Bernstein did not appear. The results are interpreted in accordance with a general model of language processing. It is suggested that in the future more attention should be paid to variables other than the purely sociological variable of social status or social class.     

Imitation improves language comprehension

Humans imitate each other during social interaction. This imitative behavior streamlines social interaction and aids in learning to replicate actions. However, the effect of imitation on action comprehension is unclear. This study investigated whether vocal imitation of an unfamiliar accent improved spoken-language comprehension. Following a pretraining accent comprehension test, participants were assigned to one of six groups. The baseline group received no training, but participants in the other five groups listened to accented sentences, listened to and repeated accented sentences in their own accent, listened to and transcribed accented sentences, listened to and imitated accented sentences, or listened to and imitated accented sentences without being able to hear their own vocalizations. Posttraining measures showed that accent comprehension was most improved for participants who imitated the speaker's accent. These results show that imitation may aid in streamlining interaction by improving spoken-language comprehension under adverse listening conditions.     

Emotion simulation during language comprehension

We report a novel finding on the relation of emotion and language. Covert manipulation of emotional facial posture interacts with sentence valence when measuring the amount of time to judge valence (Experiment 1) and sensibility (Experiment 2) of the sentence. In each case, an emotion-sentence compatibility effect is found: Judgment times are faster when facial posture and sentence valence match than when they mismatch. We interpret the finding using a simulation account; that is, emotional systems contribute to language comprehension much as they do in social interaction. Because the effect was not observed on a lexical decision task using emotion-laden words (Experiment 3), we suggest that the emotion simulation affects comprehension processes beyond initial lexical access.     

Memory mechanisms supporting syntactic comprehension

Efforts to characterize the memory system that supports sentence comprehension have historically drawn extensively on short-term memory as a source of mechanisms that might apply to sentences. The focus of these efforts has changed significantly in the past decade. As a result of changes in models of short-term working memory (ST-WM) and developments in models of sentence comprehension, the effort to relate entire components of an ST-WM system, such as those in the model developed by Baddeley (Nature Reviews Neuroscience 4: 829–839, 2003) to sentence comprehension has largely been replaced by an effort to relate more specific mechanisms found in modern models of ST-WM to memory processes that support one aspect of sentence comprehension—the assignment of syntactic structure (parsing) and its use in determining sentence meaning (interpretation) during sentence comprehension. In this article, we present the historical background to recent studies of the memory mechanisms that support parsing and interpretation and review recent research into this relation. We argue that the results of this research do not converge on a set of mechanisms derived from ST-WM that apply to parsing and interpretation. We argue that the memory mechanisms supporting parsing and interpretation have features that characterize another memory system that has been postulated to account for skilled performance—long-term working memory. We propose a model of the relation of different aspects of parsing and interpretation to ST-WM and long-term working memory.     

Embodied Cognition,affects and language comprehension

Phenomenology and the Cognitive Sciences - Our main considerations take as a starting point the educational policy demand for a model of competence of text comprehension, which should make it...     

Persistent structural priming from language comprehension to language production

To examine the relationship between syntactic processes in language comprehension and language production, we compared structural persistence from sentence primes that speakers heard to persistence from primes that speakers produced. [Bock, J. K., & Griffin, Z. M. (2000). The persistence of structural priming: transient activation or implicit learning? Journal of Experimental Psychology: General, 129, 177-192.] showed that the production of target priming structures increased the probability of spontaneously using the same structures to describe events in subsequent pictures that were semantically unrelated to the primes. These priming effects persisted across as many as ten intervening filler trials. The present studies replicated these results using auditorily presented primes to which participants only listened. The results indicated persistence of priming across all lags, with relative magnitudes of priming as large as those observed by Bock and Griffin. The implication is that structural priming is persistent regardless of the modality in which language structures are experienced, underscoring the power of priming as an implicit learning mechanism.     

Time in language: event duration in language comprehension

This work investigates how we process and represent event duration in on-line language comprehension. Specifically, it examines how events of different duration are processed and what type of knowledge underlies their representations. Studies 1–4 examined verbs and phrases in different contexts. They showed that durative events took longer to process than non-durative events and that the duration attributed to the stimulus events correlated with on-line processing times. Studies 5 and 6 indicated that durative events occur in semantically more diverse contexts and elicit semantically more diverse associations than non-durative events. Semantic and contextual diversity also correlated with attributed durations and processing times. Results indicate that (a) event-specific durations are computed on-line from multiple unfolding cues, (b) processing cost and duration representations emerge from semantic and contextual diversity reflecting our experience, and (c) key components of duration representations may be situation-specific knowledge of causal and contingency relations between events.     

Perceptual simulation in developing language comprehension

We tested an embodied account of language proposing that comprehenders create perceptual simulations of the events they hear and read about. In Experiment 1, children (ages 7–13 years) performed a picture verification task. Each picture was preceded by a prerecorded spoken sentence describing an entity whose shape or orientation matched or mismatched the depicted object. Responses were faster for matching pictures, suggesting that participants had formed perceptual-like situation models of the sentences. The advantage for matching pictures did not increase with age. Experiment 2 extended these findings to the domain of written language. Participants (ages 7–10 years) of high and low word reading ability verified pictures after reading sentences aloud. The results suggest that even when reading is effortful, children construct a perceptual simulation of the described events. We propose that perceptual simulation plays a more central role in developing language comprehension than was previously thought.     

Representing object colour in language comprehension

Embodied theories of cognition hold that mentally representing something red engages the neural subsystems that respond to environmental perception of that colour. This paper examines whether implicit perceptual information on object colour is represented during sentence comprehension even though doing so does not necessarily facilitate task performance. After reading a sentence that implied a particular colour for a given object, participants were presented with a picture of the object that either matched or mismatched the implied colour. When asked if the pictured object was mentioned in the preceding sentence, people's responses were faster when the colours mismatched than when they matched, suggesting that object colour is represented differently to other object properties such as shape and orientation. A distinction between stable and unstable embodied representations is proposed to allow embodied theories to account for these findings.     

Sensorimotor skills and language comprehension in autistic children

The objectives of this study were to examine the level of sensorimotor concepts of young autistic children and to relate these concepts to language comprehension. A sample of 16 autistic children with a mean mental age of 24.8 months was administered a standardized scale of sensorimotor intelligence and of receptive language. The autistic children demonstrated surprisingly sophisticated sensorimotor skills, particularly object permanence. While their initial performance was inferior to that of normal controls matched on mental age, particularly in their use of objects in combination, the difference between groups diminished on the second test administration. On the receptive language measure, the autistic children were less able to identify words correctly. The sensorimotor behavior of autistic children who demonstrated language comprehension did not differ from those who showed no language comprehension, except that the former group tended to use an object as an instrument somewhat more frequently. The fact that the autistic children were so impaired in language even with fairly good sensorimotor skills suggests that these skills, particularly object permanence, play a minor role in their language acquisition.Support for this research was provided by Biobehavioral Research Support Grant 516, Grant 12–41 from the March of Dimes, NIMH Postdoctoral Fellowship No. 1 F32 MH07550-01 (Ungerer), and NIMH Grant MH 33815-01. Subjects were recruited from the UCLA Research Center for the Study of Childhood Psychosis, funded by NIMH Grant MH 30897. We would like to thank members of the CRC, particularly Dr. Peter Tanguay, Dr. Barbara Fish, and Dr. B. J. Freeman, for their support of our research efforts.     

False belief and language comprehension in Cantonese-speaking children

The current research compared two accounts of the relation between language and false belief in children, namely that (a) language is generally related to false belief because both require secondary representation in a social-interactional context and that (b) specific language structures that explicitly code metarepresentation contribute uniquely to the language-false belief relation. In three studies, attempts were made to correlate Cantonese-speaking children's false belief with their general language comprehension and understanding of certain structures that explicitly express metarepresentational knowledge. Results showed that these structures failed to predict false belief after age, nonverbal intelligence, and general language comprehension were considered. In contrast, general language remained predictive of false belief after controlling for age, nonverbal intelligence, and language structures. The current findings are more consistent with a general language account than a language structure account.     

Working memory and language comprehension: A meta-analysis

This paper presents a meta-analysis of the data from 6,179 participants in 77 studies that investigated the association between working-memory capacity and language comprehension ability. A primary goal of the meta-analysis was to compare the predictive power of the measures of working memory developed by Daneman and Carpenter (1980) with the predictive power of other measures of working memory. The results of the meta-analysis support Daneman and Carpenter’s (1980) claim that measures that tap the combined processing and storage capacity of working memory (e.g., reading span, listening span) are better predictors of comprehension than are measures that tap only the storage capacity (e.g., word span, digit span). The meta-analysis also showed that math process plus storage measures of working memory are good predictors of comprehension. Thus, the superior predictive power of the process plus storage measures is not limited to measures that involve the manipulation of words and sentences.     

Prosodic phrasing is central to language comprehension

Words, like musical notes, are grouped together into phrases by their rhythmic and durational properties as well as their tonal pitch. This 'prosodic phrasing' affects the understanding of sentences. Many processing studies of prosody have investigated sentences with a single, grammatically required prosodic boundary, which might be interpreted strictly locally, as a signal to end the current syntactic unit. Recent results suggest, however, that the global pattern of prosodic phrasing is what matters in sentence comprehension, not just the occurrence or size of a single local boundary. In this article we claim that the impact of prosodic boundaries depends on the other prosodic choices a speaker has made. We speculate that prosody serves to hold distinct linguistic representations together in memory.     

Event-based plausibility immediately influences on-line language comprehension

In some theories of sentence comprehension, linguistically relevant lexical knowledge, such as selectional restrictions, is privileged in terms of the time-course of its access and influence. We examined whether event knowledge computed by combining multiple concepts can rapidly influence language understanding even in the absence of selectional restriction violations. Specifically, we investigated whether instruments can combine with actions to influence comprehension of ensuing patients of (as in Rayner, Warren, Juhuasz, & Liversedge, 2004; Warren & McConnell, 2007). Instrument-verb-patient triplets were created in a norming study designed to tap directly into event knowledge. In self-paced reading (Experiment 1), participants were faster to read patient nouns, such as hair, when they were typical of the instrument-action pair (Donna used the shampoo to wash vs. the hose to wash). Experiment 2 showed that these results were not due to direct instrument-patient relations. Experiment 3 replicated Experiment 1 using eyetracking, with effects of event typicality observed in first fixation and gaze durations on the patient noun. This research demonstrates that conceptual event-based expectations are computed and used rapidly and dynamically during on-line language comprehension. We discuss relationships among plausibility and predictability, as well as their implications. We conclude that selectional restrictions may be best considered as event-based conceptual knowledge rather than lexical-grammatical knowledge.     

Impairment of auditory perception and language comprehension in dysphasia

Men with chronic focal brain wounds were examined for their ability to discriminate complex tones, synthesized steady-state vowels, and synthesized consonant—vowel syllables. Subjects with left hemisphere damage, but not right hemisphere damage, were impaired in their ability to respond correctly to rapidly changing acoustic stimuli, regardless of whether stimuli were verbal or nonverbal. The degree of impairment in auditory processing correlated highly with the degree of language comprehension impairment. The pattern of impairment of the group with left hemisphere damage on these perceptual tests was similar to that found in children with developmental language disorders.     

Intelligibility and comprehension of time compressed sign language narratives

A method of time compressing visual displays was devised to assess the effects of compression on the comprehension of American sign language (ASL). Two ASL narratives were visually presented to three groups of deaf ASL users. A compression group was presented narratives compressed to 50% of the duration of the normal rates. A no-compression group was presented narratives signed at normal rates. A pause group was presented compressed narratives with blank pauses inserted so that the total time per narrative was equal to that of the no-compression versions. Following each narrative, subjects completed a written comprehension task. Comprehension in both compression and pause groups was significantly lower than in the no-compression group. Pauses did not significantly affect scores relative to compression. An additional group of deaf ASL users was presented an intelligibility task in which signs taken from the narratives were presented in isolation either with or without compression. The total number of signs correctly identified was significantly lower with compression. Decrements in comprehension were best explained as the result of cumulative decrements in sign intelligibility and not as the result of decrements in available processing time.     

Real-time language comprehension research using the Apple-Psych system

We describe the use of the Apple-Psych system to conduct language comprehension experiments. Using this system, we have implemented several tasks commonly used to assess the cognitive processes involved in language comprehension. These include segment-by-segment reading tasks in which the stimuli are presented visually, and divided-attention and crossmodal tasks in which the stimuli are presented both visually and auditorily. Currently we are using the system to control collection of event-related potentials while subjects comprehend auditory discourse.